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Annual  Summary 


Award  Number:  DAMD 17-02- 1-0230 

FY01  Prostate  Cancer  Research  Program  (PCRP)  Post-Doctoral  Traineeship  Award 
"Molecular  Imaging  for  IMRI-Guided  Minimally  Invasive  Treatment  of  Prostate  Cancer" 


Introduction 

Our  long-range  objective  is  to  develop  an  image-guided,  minimally  invasive  treatment  method  for  prostate 
cancer.  We  currently  use  interventional  magnetic  resonance  imaging  (iMRI)  to  guide  biopsy  needles  for 
diagnosis  and  to  guide  radiofrequency  (RF)  electrodes  for  thermal  ablation  treatment  of  cancer,  and  we  are 
planning  to  use  iMRI  to  deliver  controlled  drug  release  pellets  to  tumor.  We  use  a  low  field,  open  magnet 
system  that  allows  patient  access  during  imaging.  A  unique  aspect  of  iMRI-guided  thermal  ablation 
therapy  is  that  real  time  feedback  is  provided  from  images  of  treated  regions  and  from  non-invasive  three- 
dimensional  (3D)  temperature  measurements.  Before  this  method  can  be  extended  to  the  prostate, 
significant  engineering  developments  are  required. 

Although  MRI  provides  excellent  anatomical  detail,  it  does  not  reliably  identify  prostate  cancer. 
Fortunately,  there  are  emerging  imaging  methods  that  promise  to  improve  detection  of  prostate  tumor. 
These  include  SPECT  and  MR  spectroscopy  functional  imaging  methods  that  promise  to  help  identify 
prostate  tumor.  We  intend  to  incorporate  this  information  especially  SPECT  ProstaScint  images  into  the 
iMRI-guided  paradigm. 

Although  functional  images  promise  to  help  identify  prostate  tumor,  they  have  poor  resolution  and  do 
not  provide  the  anatomical  information  required  for  treatment  delivery.  The  solution  is  to  register  and 
superimpose  the  functional  images  with  3D  high-resolution  MR  images.  We  are  investigating  method  to 
correlate  features  from  SPECT  with  high-resolution  anatomical  MR  images  to  aid  identification  of 
prostate  cancer.  To  improve  tumor  targeting  during  treatment,  the  information  from  the  multi-modality 
will  be  integrated  with  freshly  acquired  iMRI-slice  image  to  guide  intervention  tools.  Accurate  registration 
with  anatomy  will  be  necessary  for  highly  accurate,  interventional  MRI-guided  thermal  ablations. 


Summary  Body 

As  part  of  my  postdoctoral  training,  I  am  working  with  a  team  of  researchers  to  achieve  aims.  My 
specialized  role  is  to  develop  image  registration  techniques,  but  I  participated  in  most,  if  not  all,  research 
aspects  of  molecular  imaging  for  iMRI-guided  minimally  invasive  treatment  of  prostate  cancer  (Fei  et  al., 
2001,  Fei  et  al.,  2002a,  Fei  et  al.,  2002b,  Fei  et  al.,  2002c,  Fei  et  al.,  2002d,  Fei  et  al.,  2003a,  Fei  et  al!’ 
2003b,  Fei  et  al.,  2003c,  Wilson  et  al.,  2003).  The  following  paragraphs  outline  the  training  and  research 
associated  with  the  tasks  described  in  the  Statement  of  Work. 

As  for  the  Task  1  of  image  acquisitions,  we  obtained  and  analyzed  SPECT  ProstaScint  and  high- 
resolution  MR  images  from  patients  with  verifiable  prostate  cancer  (Lee  et  al.,  2000,  Sodee  et  al.,  2001). 
We  developed  methods  for  SPECT  ProstaScint  imaging  for  the  prostate  cancer  (Sodee  et  al.,  1996,  Sodee 
et  al.,  1998,  Sodee  et  al.,  1999,  Sodee  et  al.,  2000b,  Sodee  et  al.,  2000a,  Sodee  et  al.,  2001).  We  developed 
high-resolution  3D  MR  imaging  techniques  for  the  prostate  (Fei  et  al.,  2002b,  Fei  et  al.,  2002c,  Fei  et  al., 
2003a,  Fei  et  al.,  2003b).  The  detail  descriptions  on  these  imaging  techniques  can  be  founded  in  our 
published  papers  (Fei  et  al.,  2002b,  Fei  et  al.,  2002c,  Fei  et  al.,  2003a).  I  helped  plan  image  acquisition 
methods  suitable  for  registration.  We  are  developing  choline  MR  spectroscopy  imaging  techniques 
because  of  several  technical  difficulties  such  as  MR  sequence,  image  quality,  and  imaging  time. 


As  for  the  Task  2  of  image  correlation  and  registration,  we  have  correlated  SPECT  images  with  high- 
resolution  MRI  volume  to  improve  detection  and  treatment  of  prostate  cancer  (Fei  et  al.,  2002b,  Lee  et  al., 
2000,  Lee  et  al.,  2001,  Sodee  et  al.,  2001).  First,  we  created  3D  image  registration  methods  suitable  for 
aligning  multi-modality  volumes  to  combine  anatomic  and  metabolic  information  (Fei  et  al.,  2002b,  Fei  et 
al.,  2002c,  Fei  et  al.,  2003a,  Lee  et  al.,  2000).  We  extended  our  current  mutual  information  methods  for 
robust  image  registration.  Second,  we  created  image  software  system  to  visualize  and  analyze  images  of 
the  prostate  cancer  (Fei  et  al.,  2002b,  Fei  et  al.,  2002c,  Fei  et  al.,  2003a).  Using  the  visualization  software, 
we  compared  3D  registered  image  data  using  linked  cursors  and  hand-drawn  regions-of-interest  on  side- 
by-side  image  frames.  From  a  pair  of  registered  volumes,  we  applied  multi-planar  reformatting  to 
visualize  registered  slices  from  each  of  two  volumes  using  multiple  types  of  displays.  Statistical  analyses 
were  developed  to  correlate  image  gray  values  in  ProstaScint  and  MRI  images  (Sodee  et  al.,  2000a,  Sodee 
et  al.,  2001).  In  addition  to  analysis,  we  investigated  fusion  methods  for  iMRI  targeting.  These  techniques 
are  described  in  detail  in  our  published  papers  (Fei  et  al.,  2001,  Fei  et  al.,  2002a,  Fei  et  al.,  2002b,  Fei  et 
al.,  2002c,  Fei  et  al.,  2002d,  Fei  et  al.,  2003a,  Fei  et  al.,  2003b,  Fei  et  al.,  2003c).  Third,  our  group 
compared  functional  image  data  with  histology  of  surgically  removed  prostates  (Lee  et  al.,  2000,  Lee  et 
al.,  2001,  Sodee  et  al.,  2001). 

As  for  the  training,  I  actively  participated  in  the  journal  club  every  week  as  held  in  Dr.  Wilson’s 
Laboratoiy.  I  attended  the  research  seminar  once  a  week  in  the  Biomedical  Engineering  Department.  The 
Ireland  Cancer  Center  at  our  institute,  a  NCI  Comprehensive  Cancer  Center  with  over  130  scientists  and 
physicians,  often  has  excellent  seminars  with  great  training  opportunities.  I  frequently  interact  with 
mentors  Drs.  Wilson,  Duerk,  and  Sodee.  I  attended  two  international  conferences  and  gave  two  oral 
presentations  during  the  fiscal  year. 


Key  Research  Accomplishments 

1.  We  created  a  robust  image  registration  method  for  the  combination  of  high  resolution  MRI  volume 
and  interventional  MRI  slice  (Fei  et  al.,  2001,  Fei  et  al.,  2002b,  Fei  et  al.,  2002c,  Fei  et  al.,  2002d,  Fei 
et  al.,  2003a,  Fei  et  al.,  2003b).  It  is  significant  because  it  demonstrated  feasibility  of  including  other 
image  modalities  such  as  high  resolution  MRI  and  SPECT  images  into  iMRI-guided  treatment  for 
improved  tumor  targeting. 

2.  We  created  image  registration  and  fusion  software  that  could  be  used  for  the  iMRI-guided  procedures 
(Fei  et  al.,  2002b,  Fei  et  al.,  2002c,  Fei  et  al.,  2003d,  Fei  et  al.,  2003a).  It  is  important  because  it  could 
help  physician  or  radiologist  better  locate  and  target  tumors  region  and  possibly  improve  treatment. 

3.  We  developed  MR  imaging  techniques  for  the  acquisition  of  high  resolution  prostate  images  (Fei  et 
al.,  2002b,  Fei  et  al.,  2002c,  Fei  et  al.,  2003a,  Fei  et  al.,  2003c).  Since  the  imaging  techniques  well 
delineates  the  prostate  and  its  surroundings,  they  could  possibly  be  used  for  the  prostate  cancer 
screening  and  other  clinical  applications. 

4.  We  developed  fast  imaging  techniques  for  interventional  MR  imaging  of  the  prostate  (Fei  et  al., 
2002b,  Fei  et  al.,  2002c,  Fei  et  al.,  2003a,  Fei  et  al.,  2003c).  This  is  very  important  because  the 
techniques  can  acquire  prostate  images  in  nearly  real-time  and  thus  provide  image  guidance  during 
interventional  procedures. 

5.  We  created  image  visualization  software  for  image  analysis  and  registration  evaluation  (Fei  et  al., 
2002b,  Fei  et  al.,  2002c,  Fei  et  al.,  2003a,  Fei  et  al.,  2003c).  This  byproduct  is  a  very  useful  image 
analysis  tool  with  multiple  visualization  displays.  It  also  allows  one  to  quantitatively  measure  prostate 
size,  movement,  and  deformation. 

6.  We  performed  more  than  1000  slice  to  volume  registration  experiments  using  MR  images  acquired 
from  patients  and  volunteers  (Fei  et  al.,  2002b,  Fei  et  al.,  2002c,  Fei  et  al.,  2003a,  Fei  et  al.,  2003c). 
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These  experiments  simulated  and  tested  a  variety  of  possible  conditions  in  potential  applications  and 
thus  provide  useful  information  and  guidance  for  future  clinical  applications. 

7.  We  performed  image  registration  and  fusion  experiments  using  SPECT  and  high  resolution  MR 
images  from  patients  with  prostate  cancer.  Registration  and  fusion  of  the  two  modalities  could 
possible  improve  the  diagnosis  of  prostate  tumors  (Lee  et  al.,  2000,  Lee  et  al.,  2001,  Sodee  et  al., 
2001). 

8.  We  conducted  a  variety  of  pelvic  phantom  experiments  to  test  registration  accuracy  of  high  resolution 
MR  and  SPECT  images.  These  experiments  provided  some  baseline  for  accuracy  evaluation  and 
simulation. 

9.  We  planned  animal  experiments  for  the  interventional  MRI-guided  thermal  ablation  of  the  prostate 
using  the  dog  model.  The  experiments  will  provide  some  fundamental  knowledge  for  any  future 
human  applications. 


Reportable  Outcomes 

1.  Our  paper  entitled  “Slice  to  volume  registration  and  its  potential  application  to  interventional  MRI 
guided  radiofrequency  thermal  ablation  of  prostate  cancer  (Fei  et  al.,  2003a)”  was  accepted  for 
publication  in  the  IEEE  Transaction  on  Medical  Imaging,  which  was  the  most-cited  journal  in  the 
biomedical  engineering  and  imaging  science  &  photographic  technology  categories  (See  the  citation 
webpage  of  IEEE  Transaction  on  Medical  Imaging  2003). 

2.  One  paper  entitled  “Automatic  3D  registration  for  interventional  MRI-guided  treatment  of  prostate 
cancer  (Fei  et  al.,  2002c)”  was  published  in  the  Computer  Aided  Surgery,  the  official  journal  of  the 
International  Society  for  Computer  Aided  Surgery  (ISCAS)  (See  the  webpage  of  Computer  Aided 
Surgery  2003). 

3.  One  paper  entitled  “A  comparative  study  of  warping  and  rigid  body  registration  for  the  prostate  and 
pelvic  MR  volumes  (Fei  et  al.,  2003c)”  was  published  in  the  Computerized  Medical  Imaging  and 
Graphics,  the  official  journal  of  the  Computerized  Medical  Imaging  Society,  an  international  journal 
on  imaging  and  image-archiving  in  all  medical  specialties  (See  the  webpage  of  Computerized  Medical 
Imaging  and  Graphics  2003). 

4.  One  manuscript  entitled  “Three-dimensional  MR  image  registration  of  the  prostate  and  pelvis  with 
semiautomatic  warping  (Wilson  et  al.,  2003)”  was  submitted  to  the  IEEE  Transaction  on  Medical 
Imaging  in  December  2002. 

5.  Two  conference  papers  were  published  in  the  Proceeding  of  SPIE  on  Medical  Imaging:  Visualization, 
Image  Guided  Procedures,  and  Display  (Fei  et  al.,  2003b)  and  the  Proceedings  of  the  Second  Joint 
EMBS/BMES  Conference  (Fei  et  al.,  2002d). 

6.  Two  conference  papers  are  submitting  to  two  international  conferences:  The  Sixth  International 
Conference  on  Medical  Image  Computing  and  Computer  Assisted  Intervention  2003  (Fei  et  al., 
2003d)  and  the  Second  International  Workshop  on  Biomedical  Image  Registration  (Fei  et  al.,  2003e). 

7.  I  gave  two  presentations  at  two  international  conferences,  the  2nd  Joint  International  Conference  of 
IEEE  Engineering  in  Medicine  and  Biology  Society  and  the  Biomedical  Engineering  Society  in 
Huston  in  October  2002  (Fei  et  al.,  2002d)  and  the  International  Conference  of  SPIE  on  Medical 
Imaging:  Visualization,  Image  Guided  Procedures,  and  Display  in  San  Diego  in  February  2003  (Fei  et 
al.,  2003b). 
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Conclusions 


In  the  fiscal  year,  we  fairly  well  completed  the  tasks  1  and  2  outlined  in  the  statement  of  work.  We 
achieved  nine  key  research  accomplishments  and  seven  reportable  outcomes.  We  achieved  the  goals  of 
training  and  research  in  the  first  year.  We  well  planned  the  task  3  for  the  next  year. 
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Abstract.  We  are  investigating  interventional  MRI  (iMRI)  guided 
radiofrequency  thermal  ablation  for  the  minimally  invasive  treatment  of  the 
prostate  cancer.  Nuclear  medicine  can  detect  and  localize  tumor  in  the  prostate 
not  reliably  seen  in  MRI.  We  intend  to  combine  the  advantages  of  functional 
images  such  as  SPECT  with  iMRI-guided  treatments.  Our  concept  is  to  first 
register  the  low-resolution  SPECT  with  a  high  resolution  MRI  volume.  Then  by 
registering  the  high-resolution  MR  image  with  live-time  iMRI  acquisitions,  we 
can,  in  turn,  map  the  functional  data  and  high-resolution  anatomic  information 
to  iMRI  images  for  improved  tumor  targeting.  For  die  first  step,  we  used  a  three 
dimensional  mutual  information  registration  method.  For  the  latter,  we 
developed  a  robust  slice  to  volume  (SV)  registration  algorithm  with  special 
features.  Image  data  were  from  three  patients  and  three  volunteers.  The  SV 
registration  accuracy  is  0.4  mm  ±  0.2  mm  for  transverse  slices  covering  the 
prostate.  With  our  image  registration  and  fusion  software,  simulation 
experiments  show  that  it  is  feasible  to  incorporate  SPECT  and  high  resolution 
MRI  into  the  iMRI-guided  procedures. 


1  Introduction 

We  use  an  interventional  magnetic  resonance  imaging  (iMRI)  system  to  guide 
minimally  invasive  treatments,  including  the  radioffequency  (RF)  thermal  ablation  of 
abdominal  cancers.1'3  The  iMRI  system  consists  of  a  0.2  T,  clinical  C-arm  open  MRI 
scanner,  an  in-room  RF-shielded  liquid  crystal  monitor,  an  MR  compatible  mouse,  a 
foot  pedal,  and  a  RF  device.  We  are  currently  investigating  the  extension  of  these 
techniques  to  the  treatment  of  prostate  cancer.  Since  MRI  does  not  reliably  show 
prostate  tumors,  we  intend  to  incorporate  nuclear  medicine  or  MR  spectroscopy 
images  with  higher  sensitivity  for  detecting  and  localizing  prostate  tumors.4,5  We  will 
first  register  the  low-resolution  functional  SPECT  images  with  a  high  resolution 
MRI  volume.  Then  by  registering  the  high-resolution  MR  volume  with  iMRI 
acquisitions,  we  can,  in  turn,  map  the  functional  data  and  high-resolution  anatomic 
information  to  iMRI  images  for  improved  tumor  targeting.  If  this  procedure  is 
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successful,  then  a  variety  of  potential  visualization  tools  can  help  the  physician 
appropriately  localize  and  apply  treatments. 

In  next  sections,  we  will  report  a  three  dimensional  registration  method  for  SPECT 
and  high  resolution  MRI  volumes,  a  slice  to  volume  registration  algorithm  for  iMRI 
thick  slices  and  high  resolution  MRI  volume,  and  image  registration  and  fusion 
software  for  potential  applications  in  iMRI- guided  thermal  ablation  of  the  prostate. 


2  Registration  Algorithms 


2.1  Registration  of  SPECT  and  High  Resolution  MRI  Volumes 

The  mutual  information  algorithm  was  applied  in  this  study  to  the  registration  of 
pelvic  image  volumes  from  MRI  and  SPECT  for  potential  use  in  prostate  cancer 
diagnosis,  staging,  and  treatment  planning.  Mutual  information  (MI)  registration 
method  was  chosen  because  of  its  potential  to  align  multi-modality  images13,14. 
Registration  of  ProstaScint®  SPECT  and  MR  images  is  challenging  because  the  two 
image  types  have  very  different  spatial  resolutions  and  image  features.  The  pelvic 
region  is  difficult  for  alignment  between  images  from  the  two  scans.  Before 
registration,  both  SPECT  and  MRI  volumes  were  resized  using  tri-linear  interpolation 
to  create  volumes  matrix  of  1283  with  3  mm  isotropic  voxels,  a  voxel  size  between 
that  of  the  two  scans.  The  standard  parameter  set  for  automatic  registration  included: 
256  intensity  levels  for  each  volume,  the  entire  2D  joint  histogram,  the  full  field  of 
view  of  1283  voxels  for  both  volumes,  and  no  masking  or  cropping  of  either  volume. 


2.2  Registration  of  iMRI  Slice  and  High  Resolution  MRI  Volume 

We  used  two  similarity  measures,  mutual  information6  and  correlation  coefficient 
(CC),  in  our  registration.  We  used  a  similar  algorithm  as  previously  reported  by  us.8 
We  use  a  multi-resolution  approach  and  perform  registration  from  low  to  high 
resolution.  We  use  correlation  coefficient  at  the  two  lower  resolutions  because  it  gives 
fewer  local  maximums  and  because  it  can  be  calculated  faster  than  MI.9  We  use  MI  at 
full  resolution  because  the  peaked  similarity  function  gives  a  more  precise  solution 
than  CC.10  To  avoid  local  maximums,  we  include  a  restarting  feature  where 
registration  is  restarted  with  randomly  perturbed  parameters  obtained  from  a  uniform 
distribution  about  the  initial  transformation  values  at  the  current  resolution  being 
used.11,12  The  algorithm  restarts  until  the  absolute  CC  is  above  an  experimentally 
determined  threshold  or  the  maximum  number  of  restarts  is  reached. 
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3  Experimental  Methods 


3.1  SPECT  Image  Acquisition 

The  study  includes  three  patients  S1-S3  with  either  high  Gleason  scores  (>5)  from 
biopsy  or  rising  PSA  level  (>10  mcg/L,  prostate  specific  antigen)  or  palpatation 
staging  beyond  stage  Tl.  After  patient  eligibility  was  established,  patients  gave 
informed  consent.  The  Institutional  Review  Board  of  the  University  Hospitals  of 
Cleveland  approved  the  imaging  protocol.  The  radiotracer  used  for  SPECT  imaging  is 
ProstaScint®  (Cytogen  Corporation,  Princeton,  NJ),  a  monoclonal  antibody  that  binds 
to  prostate-specific  membrane  antigen  (PSMA). 

Approximately  four  days  after  injecting  5  mCi  ProstaScint®,  the  abdominal  region 
and  pelvic  region  were  scanned  using  a  two-head  Siemens  E.CAM*  camera  (Siemens 
Medical  System,  Inc.,  Hoffman  Estates,  Illinois,  USA).  The  evening  before  the 
scanning,  patients  were  asked  to  perform  a  bowel  prep  with  Fleet  R  Prep  Kit  #3  (Fleet 
Pharmaceuticals,  Lynchburg,  VA).  Images  were  acquired  with  a  medium  energy 
collimator  and  15%  energy  window.  The  acquisition  parameters  included  a  step-and- 
shoot  motion,  a  128  x  128  pixel  matrix  for  each  projection,  an  imaging  time  of  25  sec 
per  stop,  and  a  total  of  120  stops  over  a  full  360°  rotation.  The  field  of  view  of  was 
53.3  x  38.7  cm2.  The  iterative  image  reconstruction  algorithm  OSEM  (Ordered 
Subsets  Expectation  Maximization)  1 '  was  used  to  reconstruct  the  SPECT  images 
with  8  subsets  and  10  iterations.  The  SPECT  images  were  comprised  of  4.8  mm3 
isotropic  voxels.  Each  patient  had  one  SPECT  scan  of  the  pelvis. 


3.2  High  Resolution  MR  Image  Acquisitions 

High-resolution  MRI  volumes  were  acquired  using  a  1.5  T  Siemens  MRI  system 
(Magnetom  Symphony,  Siemens  Medical  Systems,  Erlangen,  Germany).  An  8- 
element  phased  array  body  coil  was  used  to  ensure  coverage  of  the  prostate  with  a 
uniform  sensitivity.  Typically  two  anterior  and  two  posterior  elements  were  enabled 
for  signal  acquisition.  We  used  two  different  MR  sequences.  First,  we  used  a  3D  rapid 
gradient  echo  sequence  (PSIF)  designed  to  acquire  the  spin-echo  component  of  the 
steady  state  response,  rather  than  the  free  induction  decay.  The  spin  echo  component 
forms  immediately  prior  to  the  RF  pulse;  it  is  shifted  toward  the  prior  RF  pulse 
through  appropriate  gradient  waveform  design.  The  sequence  with  9.4/5.0/60 
(TR/TE/flip)  yields  160  x  256  x  128  voxels  over  a  219  x  350  x  192-mm  rectangular 
FOV  and  1.4x  1.4  x  1.5-mm  voxels  oriented  to  give  the  highest  resolution  for 
transverse  slices.  There  is  over  sampling  at  31%  in  the  slice  direction  to  reduce 
aliasing  artifacts.  The  acquisition  time  is  4  min  and  15  sec.  This  sequence  gave 
excellent  image  contrast  for  the  prostate  and  its  surroundings.  It  was  used  to  acquire 
volumes  for  patients  SI -S3.  Second,  we  used  a  3D  RF  spoiled  gradient  echo  steady 
state  pulse  sequence  (FLASH)  with  TR/TE/flip  parameters  of  12/5.0/60  which  give 
256  x  256  x  128  voxels  over  a  330  x  330  x  256-mm  field  of  view  (FOV)  to  yield 

1.3  x  1.3  x  2.0-mm  voxels  oriented  to  give  the  highest  resolution  for  transverse  slices. 
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The  acquisition  time  is  5  min  and  38  sec.  This  sequence  is  good  for  pelvic  imaging 
but  is  not  ideal  for  the  prostate.  It  was  used  to  acquire  volumes  for  volunteers  S4-S6. 

When  acquiring  MR  volumes,  volunteers  laid  supine  in  a  manner  similar  to  the 
diagnostic  position  in  routine  MR  scanning.  Between  volume  acquisitions,  volunteers 
got  up  from  the  MR  table,  stretched,  and  walked  around  to  ensure  that  they  would 
assume  a  different  position  when  they  laid  back  on  the  table.  The  coil  array  was 
centered  on  the  prostate.  We  acquire  one  volume  from  each  of  patients  S1-S3  and 
three  volumes  from  each  of  volunteers  S4-S6. 


3.3  Interventional  MRI  Image  Acquisitions  and  Simulation 

We  acquired  iMRI  images  using  a  clinical  0.2  T  C-arm  open  MR  scanner  (Siemens 
Open  Symphony,  Erlangen,  Germany)  modified  for  interventional  MRI  procedures 
and  in  this  paper  referred  to  as  the  iMRI  system.  We  used  a  3D  PSIF  with  25/13/60 
(TR/TE/FA)  for  image  volume  acquisitions  and  two-dimensional  (2D)  PSIF  with 
15.2/7.4/45  (TR/TE/FA)  for  image  slice  acquisitions.  The  iMRI  volumes  were 
256x256x100  with  voxel  size  of  1.3x1. 3x1. 4  mm.  The  iMRI  slices  were  128x128 
with  in-plane  pixel  size  of  2.8x2.8  mm  and  with  effective  slice  thickness  of  5  mm. 

We  acquired  iMRI  images  under  the  conditions  simulating  the  treatment 
application.  The  volunteer  was  supine,  and  his  legs  were  supported  at  30°-60°  relative 
to  the  horizon  and  separated  in  a  “V”  with  an  angle  of  60°-90°  between  two  legs.  This 
is  similar  to  the  lithotomy  position  used  in  prostate  therapies,  and  it  should  provide 
access  for  needle  insertion  in  brachytherapy  or  RF  thermal  ablation.  We  call  this  the 
treatment  position.  For  each  of  the  volunteers  S4-S6,  we  acquired  two  volumes  and  30 
iMRI  image  slices  covering  the  prostate.  They  included  10  transverse,  10  coronal,  and 
10  sagittal  image  slices.  We  call  these  images  “actual”  iMRI  images  to  differentiate 
them  from  later  experiments  using  “simulated”  iMRI  slices. 

To  test  a  variety  of  clinical  conditions,  we  used  high-resolution  MRI  volumes  to 
simulate  iMRI  images  by  creating  thick  slices  and  adding  noise  and  receive  coil 
inhomogeneity.  Clinically,  we  typically  use  an  iMRI  slice  thickness  of  4.0  -  6.0  mm. 
We  averaged  3  slices  1.4  mm  thick  to  create  a  4.2  mm  thick  slice.  We  added  noise  to 
the  simulated  iMRI  image.15,16 


3.4  Registration  Experiments 

We  performed  three  dimensional  MI  registration  of  MRI  and  SPECT  images  using 
three  pairs  of  data  sets  from  patients  S1-S3.  Before  registration,  we  preprocessed  both 
MR  and  SPECT  images  such  as  intensity  scaling  and  isotropic  processing. 

We  used  nine  pairs  of  high-resolution  MR  volumes  to  perform  simulated  slice  to 
volume  registration  experiments.  For  each  volume  pair,  we  extracted  data  from  one 
volume  to  simulate  thick  iMRI  image  slices;  and  then  we  registered  the  simulated 
image  slices  to  the  other  volume.  We  desire  an  iMRI  slice  image  acquisition  method 
that  gives  robust,  accurate  registrations  and  is  relatively  insensitive  to  acquisition 
parameters.  Hence,  we  performed  experiments  to  determine  the  dependence  on  slice 
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orientation  (transverse,  sagittal  and  coronal),  on  slice  position  relative  to  the  prostate 
(above,  centered,  and  below)  and  on  image  noise  from  fast  imaging  techniques. 

We  performed  SV  registration  experiments  using  actual  iMRI  images.  We  visually 
evaluated  results.  For  each  volunteer  S4-S6,  there  were  three  high-resolution  MR 
volumes  and  30  iMRI  image  slices  giving  90  SV  registration  experiments,  and  a  total 
of  270  experiments. 


3.5  Registration  Evaluation 

We  evaluated  registration  experiments  by  visual  inspection.  We  used  RegViz,  a 
program  created  in  IDL  in  our  laboratory  with  multiple  visualization  and  analysis 
methods.  First,  we  manually  segmented  prostate  boundaries  in  image  slices  and 
copied  them  to  corresponding  slices.  This  enabled  visual  determination  of  the  overlap 
of  prostate  boundaries  over  the  entire  volume.  Second,  color  overlay  displays  were 
used  to  evaluate  overlap  of  structures.  To  visualize  potential  differences,  it  was  quite 
useful  to  interactively  change  the  contribution  of  each  image  using  the  transparency 
scale.  Third,  we  used  a  sector  display,  which  divided  the  reference  and  registered 
images  into  rectangular  sectors  and  created  an  output  image  by  alternating  sectors 
from  the  two  input  images.  Even  subtle  shifts  of  edges  would  be  clearly  seen. 

The  success  of  registration  between  SPECT  and  MRI  was  determined  by 
comparing  computed  registration  results  to  manual  registration,  our  gold  standard. 
Manual  registration  was  done  by  two  board-certified  nuclear  medicine  radiologists 
who  were  blinded  to  the  automatic  registration  results  prior  to  performing  registration. 
Manual  registration  was  done  using  a  software  package  with  a  graphical  user  interface 
(GUI)  developed  in-house,  which  allows  graphical  manipulation  of  volumes  with  six 
degrees  of  freedom  in  a  rigid  body  registration.  A  successful  automatic  registration  is 
defined  as  when  the  transformation  parameters  are  almost  equal  to  that  of  the  manual 
registration.  Displacements  must  be  <  2  voxels  (6  mm)  in  the  x,  y,  or  z  directions  and 
<  2  degree  in  rotation  for  the  three  angles  measured  about  each  of  the  3  axes. 

Our  standard  evaluation  method  for  slice  to  volume  registration  was  to  compare 
SV  and  volume  to  volume  (W)  registration.18  For  volume  pairs  acquired  over  a  short 
time  span  from  a  supine  subject  with  legs  flat  on  the  table,  prostates  were  well  aligned 
and  prostate  centroid  displacements  were  typically  <  1  mm.  The  W  registration 
accuracy  as  determined  from  displacements  of  pelvic  bony  landmarks  was 

1.6  ±  0.2  mm.  To  compare  SV  and  W  registration,  we  defined  a  rectangular  volume 
of  interest  (VOI)  just  covering  the  prostate  and  calculated  voxel  displacements 
between  the  two  registrations.  We  defined  the  SV  registration  as  being  successful 
when  the  3D  displacement  was  less  than  2.0  mm. 
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4  Results 


4.1  Registration  of  iMRI  Slice  to  High  Resolution  MRI  Volume 

We  determined  SV  registration  results  for  slices  near  the  prostate  in  the  three  standard 
orthogonal  orientations.  Comparing  to  W,  mean  and  standard  deviation  registration 
errors  across  9  volume  pairs  and  60  SV  registration  experiments  were  0.4  mm  ±  0.2 
mm,  0.5  mm  ±0.2  mm,  and  2.6  mm  ±  1.6  mm  for  transverse,  coronal  and  sagittal 
slices  covering  the  prostate,  respectively.  Transverse  slices  worked  best  because  they 
contain  many  relatively  rigid  anatomical  structures. 

Simulation  experiments  showed  SV  registration  to  be  very  insensitive  to  noise.  We 
performed  over  100  registration  experiments  with  noise  added  to  give  signal  to  noise 
ratio  (SNR)  ranging  from  20  to  5.  Using  the  slice  configurations  recommended  above 
(transverse  slices  near  the  prostate  center),  we  obtained  100%  successful  registrations 
(an  error  <  2.0  mm)  for  SNR’s  =  10,  a  value  much  worse  than  the  clinical  SNR  value 
of  ~  25  on  our  iMRI  system. 

As  for  SV  registration  of  actual  iMRI  image  slices  with  a  high-resolution  MR 
volume,  the  contours  overlap  and  overlay  images  show  that  the  prostate  matches  very 
well.  Other  visual  inspection  techniques  also  demonstrate  excellent  registration.  Note 
that  a  single  iMRI  image  was  used  to  produce  this  registration  result. 

We  now  describe  some  aspects  of  the  implementation.  The  time  for  an  SV 
registration  was  typically  about  5  sec  on  a  Pentium  IV,  1.8  GHz  CPU,  with  1  Gbytes 
of  memory.  The  algorithm  was  written  in  IDL  (Interactive  Data  Language,  Research 
System  Inc.,  CO)  and  could  probably  be  made  much  faster  in  a  lower  level  language 
such  as  C. 


4.2  Registration  Results  of  SPECT  and  High  Resolution  MRI  Volumes 

An  example  of  a  successful  automatic  registration  for  Patient  S4  is  shown  in  Figure  1. 
All  anatomical  features  including  the  bone  marrow  in  the  femur  and  pubic  symphysis 
are  well  aligned  in  the  color  overlay.  Using  the  standard  algorithm  in  Section  2.1,  the 
MI  program  successfully  registered  the  three  volume  pairs.  The  standard  parameter 
set  was  defined  above  with  the  only  exception  that  the  lower-left  quadrant  of  the  joint 
histogram  was  used  for  calculating  MI.  A  successful  registration  was  judged  by  the 
.criteria  defined  previously. 


4.3  Image  Fusion  and  Visualization 

We  created  image  registration  and  fusion  software  for  the  potential  applications  in 
iMRI-guided  procedures  (Fig.  1).  Before  treatment,  we  preprocess  SPECT  and  high 
resolution  MRI  volumes,  such  as  creating  isotropic  volumes  and  cropping.  Then,  we 
register  them  and  transfer  the  pair  of  aligned  data  set  to  the  computer  for  slice  to 
volume  registration  late.  During  treatment,  we  freshly  acquire  iMRI  image  slices  for 
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the  guidance  of  needle  insertion.  To  target  tumors,  we  need  the  information  from 
SPECT  images.  After  pressing  a  button,  the  program  automatically  registers  iMRI 
slices  with  the  high  resolution  MRI  volume  and  thus  also  matches  them  with  SPECT 
images.  The  registered  images  are  shown  in  the  three  windows  at  the  top  line  (Fig.  1). 
After  registration,  the  program  creates  fused  images  as  displayed  at  the  bottom. 


w  .  .  l&toitoSk* 

J  U .  J  jJ  „J  4 _ 


Fig.  I.  User  interface  of  the  image  registration  and  fusion  software.  The  top  three  windows 
from  left  to  right  show  corresponding  registered  SPECT,  high  resolution  MRI,  and  iMRI 
images,  respectively.  The  bottom  three  windows  from  left  to  right  show  the  fused  images, 
iMRI/MRI,  SPECT/MRI,  and  SPECT/iMRI,  respectively.  Other  buttons  and  sliders  control 
the  configuration  and  registration.  High  resolution  MRI  and  SPECT  images  are  from  the 
patient  SI  and  the  iMRI  image  was  simulated  from  the  high  resolution  MRI  image. 


5  Discussions  and  Conclusion 

The  registration  experiments  presented  here  provided  fairly  comprehensive  tests  for 
the  potential  application  in  iMRI-guided  RF  thermal  ablation  of  the  prostate.  The  slice 
to  volume  registration  algorithm  was  extremely  robust  to  noise  levels,  far  beyond 
those  encountered  in  clinical  iMRI  applications.  The  actual  iMRI  images  acquired 
under  more  realistic  conditions  further  tested  practicality.  We  recommend  that  image 
data  are  obtained  under  comparable  conditions  by  keeping  a  similar  posture  and  by 
taking  clinical  measures  to  reduce  rectal  and  bladder  filling.  For  images  acquired  in 
quite  different  positions  with  significant  deformation,  we  previously  reported  a 
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warping  registration  method  19,20  that  can  correct  the  deformations  at  the  expense  of 
additional  complexity,  time,  and  possibly  robustness. 

Finally,  we  conclude  that  the  slice  to  volume  registration  algorithm  is  quite  robust 
for  transverse  slice  images  covering  the  prostate  and  that  registration  accuracy  is 
probably  sufficiently  accurate  to  aid  iMRI-guided  thermal  ablation  of  prostate  cancer. 
It  is  quite  feasible  to  include  previously  acquired  high-resolution  MRI  and  nuclear 
images  into  iMRI-guided  treatment  procedures.  We  are  beginning  to  explore  this 
application  in  animal  experiments. 
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Procedures:  Similarity  Measurements,  Interpolation 
Methods,  and  Applications  to  the  Prostate 
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Abstract.  We  are  investigating  interventional  MRI  (iMRI)  guided 
radiofrequency  thermal  ablation  for  radiofrequency  thermal  ablation  treatment 
of  prostate  cancer.  Nuclear  medicine  and  MR  spectroscopy  can  detect  and 
localize  tumor  in  the  prostate  not  reliably  seen  in  MR.  We  are  investigating 
methods  to  combine  the  advantages  of  functional  images  such  as  SPECT  with 
iMRI-guided  treatments.  Our  concept  is  to  first  register  die  low-resolution 
functional  images  with  a  high  resolution  MR  volume.  Then  by  registering  the 
high-resolution  MR  image  with  live-time  iMRI  acquisitions,  we  can,  in  turn, 
map  the  functional  data  and  high-resolution  anatomic  information  to  iMRI 
images  for  improved  tumor  targeting.  In  this  study,  we  registered  noisy,  thick 
iMRI  image  slices  with  high-resolution  MR  volumes  and  called  this  slice-to- 
volume  registration.  We  investigated  three  similarity  measures,  i.e.,  normalized 
mutual  information,  mutual  information,  and  correlation  coefficient;  and  three 
interpolation  methods,  i.e.,  re-normalized  sine,  tri-linear,  and  nearest  neighbor. 
After  comparison,  we  created  a  robust  registration  method  for  this  application. 
Over  300  registration  experiments  showed  that  transverse  slice  images  covering 
the  prostate  work  best  with  a  registration  error  of”  0.5  mm. 


1  Introduction 

We  are  investigating  interventional  MRI  (iMRI)  guided  radiofrequency  (RF)  thermal 
ablation  for  die  minimally  invasive  treatment  of  prostate  cancer.  Nuclear  medicine 
and  MR  spectroscopy  can  detect  and  localize  tumor  in  the  prostate  not  reliably  seen  in 
MR.  We  are  investigating  methods  to  combine  the  advantages  of  functional  images 
such  as  SPECT  with  iMRI-guided  treatments.  Our  concept  is  to  first  register  the  low- 
resolution  functional  images  with  a  high  resolution  MRI.  Then  by  registering  the 
high-resolution  MR  volume  with  live-time  iMRI  acquisitions,  we  can,  in  turn,  map 
the  functional  data  and  high-resolution  anatomic  information  to  iMRI  images  to  aid 
tumor  targeting. 
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In  this  study,  we  are  investigating  methods  to  register  live-time  iMRI  image  slices 
with  a  previously  obtained,  high  resolution  MRI  volume.  We  call  this  slice-to-volume 
(SV)  registration.  Previously  reported  methods  for  SV  registration  were  mainly 
applied ^  to  the  brain  for  applications  of  functional  MRI,1*2  postmortem  pathology 
studies,3  and  anatomical  modeling.4  In  this  report,  we  compared  three  interpolation 
methods  and  three  similarity  measures  for  this  application.  We  registered  noisy,  thick 
iMRI  image  slices  with  high-resolution  MR  volumes.  Over  300  registration 
experiments  were  performed  with  9  pairs  of  MR  volume  images  acquired  from  three 
health  volunteers. 


2  Registration  Algorithms 


2.1  Similarity  Measurements 


We  used  three  similarity  measures,  normalized  mutual  information  (NMI),  mutual 
information  (MI),  and  correlation  coefficient  (CC),  in  our  registration.  One  image  R  is 
the  reference,  and  the  other  F  is  floating.  Their  mutual  information  MI(R,F)  is  given 
below.5 

log  ^(r>/> 

rj  PR(r)'PF(f) 

The  joint  probability  /^(r,/)  and  the  marginal  probabilities  Pj{(r)  of  the  reference 
image  and  pF(f)  of  the  floating  image,  can  be  estimated  from  the  normalized  joint 
intensity  histograms.  When  two  images  are  geometrically  aligned,  MI  is  maximal.5 
We  used  the  NMI  version  proposed  by  Maes.5 


NMI(R,F)  = 


2MI(R,F) 
H(R)  +  H(F) 


where  H(R)  =  -Jdplt(r)logpR(r)  and  H(F)=-^pF(/)logpF(f)  ■ 
r  / 

The  correlation  coefficient  CC(R,F)  is  given  below.6 

CC(RF)  =  ^  Z(R(r)-R(r))(F(f)-F(f)) 

VX(*(r)  -*(r)J*£(F(/)  -  ?(/■))* 

Here  R(r),  F(f)  denote  the  average  intensities  of  the  reference  and  floating  images 
and  the  summation  includes  all  voxels  within  the  overlap  of  both  images. 
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2.2  Interpolation  Methods 

We  used  three  interpolation  methods,  i.e.,  nearest  neighbor  (NN),  tri-linear,  and  re¬ 
normalized  sine  interpolation.7,8  The  conventional  sine  interpolation  with  a  cosine 
Hamming  window  is  described  as  below.7 

I'(x,y,z)  =  XXX/(-y>  Y.Z)  ■  HS(X>  X>R) '  HS(y,  Y,R)  ■  HS(z,Z,R ) 

X  Y  Z 

with 

HS(a,  A ,  R)  =  •  {  1  +  cos[;r(a  -  A)  I R  +  lj}> 

where  Xt  Y,  Z,  are  the  coordinates  of  the  original  data  set  (7);  x,  y,  z  are  the  coordinates 
of  the  re-formatting  data  set  (/' );  A  is  a  symbol  representing  X,  Y,  or  Z,  and  a 
represents  x,  y9  or  z;  and  R  is  the  kernel  size.  The  Hamming  function  eliminates 
problems  with  oscillatory  effects  at  discontinuities  and  guarantees  that  the 
convolution  coefficients  fall  of  to  zero  at  the  edge  of  the  sine  kernel  (i.e.,  at  fa/=R+J, 
where  R~5  in  this  study).7 

A  previous  report  showed  that  re-normalizing  the  interpolation  kernel  for  a 
constant  integral  could  make  significant  improvement  in  performance  of  the 
conventional  sine  interpolation.7  In  our  implementation,  we  used  the  re-normalized 
sine  interpolation  method.7  We  replaced  HS  in  the  above  equation  with 

HS'ioc,  A ,  R)  =  HS(o A,  R)/%HS(a,  A ,  R)  =  HS(a,  A ,  R)/\HS(a)\  • 


2.3  Comparison  of  Different  Similarity  Measures  and  Interpolation  Methods 

We  compared  three  similarity  measures,  NMI,  MI,  and  CC,  at  different  resolutions  in 
order  to  determine  their  suitability  for  SV  registration.  At  1/4  resolution ,  we 
resampled  images  so  as  to  give  1/4  number  of  the  voxels  along  each  linear  dimension. 
At  full  resolution ,  we  used  the  full  number  of  voxels.  We  plot  the  similarity  measures 
as  a  function  of  translations.  After  two  typical  high-resolution  MR  volumes  were 
registered,9  values  were  plotted  with  the  origin  as  the  optimal  transformation.  We 
calculated  similarity  values  while  moving  the  simulated  iMRI  image  relative  to  the 
high-resolution  MR  image  along  coronal  (anterior-posterior)  and  sagittal  (left-right) 
axis.  When  obtaining  floating  images,  we  used  the  three  different  interpolation 
methods. 

At  1/4  resolution  (Fig.  1),  CC  surfaces  are  much  smoother  than  NMI  and  MI, 
which  are  noisy  and  contain  a  false  global  maximum  that  could  lead  to  a  false  answer 
and  many  local  maxima.10  From  these  figures,  we  infer  that  CC  is  better  at  low 
resolution.  Comparing  CC  surfaces  of  different  interpolations,  sine  and  tri-linear  have 
similar  surfaces,  and  tri-linear  is  better  than  nearest  neighbor.  For  this  application,  we 
chose  tri-linear  interpolation  instead  of  sine  because  it  is  much  faster  and  because  it 
has  comparable  performance.  Finally,  features  of  CC  and  tri-linear  demonstrate  their 
most  suitability  at  low  resolution. 

At  full  resolution  (Fig.  2),  NMI  and  MI  surfaces  are  much  peaked  than  CC  that  are 
very  good  for  accurate  optimization,  but  once  again  there  is  high  frequency  noise  in 
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the  NMI  and  MI  curves,  far  from  the  optimum,  that  gives  rise  to  local  maxima  that 
must  be  avoided.  Comparing  three  interpolation  methods,  sine  gave  the  sharpest  peak 
at  the  optimum;  nearest  neighbor  interpolation  gave  a  flat  peak  with  a  width  of  one 
voxel;  and  tri-linear  gave  a  result  between  the  other  two.  As  stated  above,  tri-linear  is 
much  faster  that  sine  with  similar  performance.  NMI  and  MI  have  no  significant 
difference  but  MI  is  a  little  bit  faster  than  NMI  in  our  implementation.  These  analyses 
show  that  MI  and  tri-linear  are  most  appropriate  at  the  full  resolution. 


Sine  Tri-linear  Nearest  Neighbor 


Fig.  1  Similarity  surfaces  are  plotted  as  a  function  of  translations  at  the  1/4  resolution  in  the 
multi-resolution  registration  process.  Two  high-resolution  MRI  volumes  were  registered,  and 
they  are  down  sampled  by  1/4  along  each  linear  dimension,  giving  a  distance  between  voxel 
centers  of  «  5.5  mm.  From  die  optimal  parameters,  we  computed  die  similarity  values  of  the 
simulated  iMRI  and  MRI  images  as  a  function  of  translations  along  the  coronal  (anterior- 
posterior)  and  sagittal  (left-right)  axis.  From  top  to  bottom,  normalized  mutual  information 
(NMI),  mutual  information  (MI),  and  correlation  coefficient  (CC)  surfaces  are  plotted.  From 
left  to  right,  sine,  tri-linear,  and  nearest  neighbor  interpolations  are  used  to  obtain  the  floating 
images,  respectively.  The  noisey  NMI/MI  surfaces  show  a  false  global  maximum  and  many 
local  maxima.  CC  surfaces  are  much  smoother  indicating  its  suitability  for  low  resolution. 
Nearest  neighbor  has  a  flat  peak  with  a  width  of  one  voxel  in  similarity  surfaces.  Images  are 
from  volunteer  S2. 


2.4  Rigid  Body  Registration  Algorithm  with  Special  Features 

The  algorithm  includes  special  features  to  improve  robustness  for  registration  of  MR 
prostate  images.9  Suppose  the  iMRI  image  slice  is  the  reference  image,  the  matching 
slice  extracted  from  the  high-resolution  MRI  volume  is  the  floating  image.  We  use  a 
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multi-resolution  approach  and  perform  registration  from  low  to  high  resolution.  We 
use  CC  at  the  two  lower  resolutions  because  it  gives  fewer  local  maxima  and  because 
it  can  be  calculated  faster  than  Ml.  We  use  MI  at  full  resolution  because  of  its  peaked 
surface.  To  avoid  local  maxima,  we  include  a  restarting  feature  where  registration  is 
restarted  with  randomly  perturbed  parameters  obtained  from  a  uniform  distribution 
about  the  initial  transformation  values  at  the  current  resolution  being  used.  The 
algorithm  restarts  until  the  absolute  CC  is  above  a  threshold  of  0.5  as  experimentally 
determined  or  the  maximum  number  of  restarts  is  reached.  Absolute  CC  is  used  rather 
than  MI  because  it  has  a  well-defined  range  between  0  and  1  and  because  it  provides 
an  independent  check  of  the  MI  result  at  the  highest  resolution.11,12 


NM( 


CC 


Sine  Tri-linear  Nearest  Neighbor 

Fig.  2  Similarity  functions  are  plotted  as  a  function  of  translations  at  full  resolution.  Many 
details  are  given  in  the  legend  of  Fig.  1 .  NMI  and  MI  surfaces  are  much  peaked  than  CC, 
especially  with  sine  and  tri-linear  interpolation.  The  voxel  is  isotropic  with  1.4  mm  on  a  side. 
Image  data  are  the  same  used  in  Fig.  1. 

We  record  all  important  results  following  an  optimization  cycle  including  the  CC 
and/or  MI  values  and  the  transformation  parameters.  At  the  end  of  processing  at  a 
lower  resolution,  we  always  select  the  transformation  parameters  having  the 
maximum  CC  value.  We  then  scale  the  translation  parameters  appropriately  and 
assign  the  new  parameters  to  be  initial  values  at  the  next  higher  resolution.  At  the 
highest  resolution,  MI  instead  of  CC  is  the  similarity  measure,  and  we  select  the  final 
transformation  parameters  to  be  those  having  the  maximum  MI  value. 

Additional  algorithm  details  are  now  described.  For  registration,  we  use  rigid  body 
transformation  (three  translations  and  three  rotations).  For  optimization,  we  use  the 
downhill  simplex  method  of  Nelder  and  Mead.13  Optimization  of  similarity  ends 
either  when  the  maximum  number  of  calculations  is  reached  (typically  500)  or  the 
fractional  change  in  the  similarity  function  is  smaller  than  a  tolerance  (typically 
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0.001).  The  input  MRI  volume  is  a  3D  MR  acquisition  giving  256  x  256  x  128  nearly 
isotropic  voxels  over  a  field  of  view  covering  the  whole  pelvis.  We  create  isotropic 
voxels  of  about  1.4  mm  on  a  side  using  3D  linear  interpolation.  We  use  IDL 
(Interactive  Data  Language,  Research  System  Inc.,  Boulder,  CO.)  as  the  programming 
language.  We  use  an  initial  guess  assuming  an  identity  transformation,  i.e.,  all  initial 
translation  and  rotation  parameters  are  zero,  because  the  patient  is  normally  oriented 
approximately  the  same  way  from  one  scan  to  the  next.  We  set  the  maximum  numbers 
of  restarts  at  10,  5,  and  3,  from  low  to  high  resolution,  respectively. 


2.5  Simulation  of  iMRI  Slice  Images 

We  used  high-resolution  MRI  volumes  to  simulate  iMRI  images  by  creating  thick 
slices  and  adding  noise  and  receive  coil  inhomogeneity.  Clinically,  we  typically  use 
an  iMRI  slice  thickness  of  4.0  -  6.0  mm.  We  averaged  3  slices  1.4  mm  thick  to  create 
a  4.2  mm  thick  slice. 

We  added  noise  to  the  simulated  iMRI  image.  MR  noise  is  described  by  the  Rician 
distribution,14  but  at  reasonably  high  signal  values,  the  noise  is  accurately 
approximated  with  Gaussian  white  noise.15  We  measured  typical  SNR  on  our  open 
magnet  system  using  a  homogenous  phantom  and  methods  described  elsewhere.16  We 
then  added  Gaussian  noise  to  the  simulated  iMRI  slice  images  either  to  match  the 
measured  SNR  or  to  give  much  greater  noise  to  further  stress  registration.  We  report 
noise  experiments  using  the  SNR  of  the  simulated  slice  images. 


2.6  Registration  Evaluation 

We  evaluated  registration  experiments  by  visual  inspection.  We  used  RegViz,  a 
program  created  in  IDL  in  our  laboratory  with  multiple  visualization  and  analysis 
methods.  First,  we  manually  segmented  prostate  boundaries  in  image  slices  and 
copied  them  to  corresponding  slices.  This  enabled  visual  determination  of  the  overlap 
of  prostate  boundaries  over  the  entire  volume.  Second,  color  overlay  displays  were 
used  to  evaluate  overlap  of  structures.  To  visualize  potential  differences,  it  was  quite 
useful  to  interactively  change  the  contribution  of  each  image  using  the  transparency 
scale.  Third,  we  used  a  sector  display,  which  divided  the  reference  and  registered 
images  into  rectangular  sectors  and  created  an  output  image  by  alternating  sectors 
from  the  two  input  images.  Even  subtle  shifts  of  edges  would  be  clearly  seen. 

Our  standard  evaluation  method  for  slice  to  volume  registration  was  to  compare 
SV  and  W  registration.9  For  volume  pairs  acquired  over  a  short  time  span  from  a 
supine  subject  with  legs  flat  on  the  table,  prostates  were  well  aligned  and  prostate 
centroid  displacements  were  typically  <  1  mm.  The  registration  accuracy  as 
determined  from  displacements  of  pelvic  bony  landmarks  was  1.6  ±0.2  mm.  To 
compare  SV  and  W  registration,  we  defined  a  rectangular  volume  of  interest  (VOI) 
just  covering  the  prostate  and  calculated  voxel  displacements  between  the  two 
registrations.  We  defined  the  SV  registration  as  being  successful  when  the  3D 
displacement  was  less  than  2,0  mm. 
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3  Experimental  Methods 


3.1  Image  Acquisition 

All  MRI  volumes  were  acquired  using  a  1.5  T  Siemens  MRI  system  (Magnetom 
Symphony,  Siemens  Medical  Systems,  Erlangen,  Germany).  An  8-element  phased 
array  body  coil  was  used  to  ensure  coverage  of  the  prostate  with  a  uniform  sensitivity. 
Typically  two  anterior  and  two  posterior  elements  were  enabled  for  signal  acquisition. 
We  used  a  3D  rapid  gradient  echo  sequence  (PSIF)  designed  to  acquire  the  spin-echo 
component  of  the  steady  state  response,  rather  then  the  free  induction  decay.  The  spin 
echo  component  formed  immediately  prior  to  the  RF  pulse,  and  it  was  shifted  toward 
the  prior  RF  pulse  through  appropriate  gradient  waveform  design.  The  sequence  with 
9.4/5.0/60  (TR/TE/flip)  yielded  160  x  256  x  128  voxels  over  a  219  x  350  x  192-mm 
rectangular  FOV  and  1.4  x  1.4  x  1.5-mm  voxels  oriented  to  give  the  highest  resolution 
for  transverse  slices.  There  was  over  sampling  at  31%  in  the  slice  direction  to  reduce 
aliasing  artifacts.  The  acquisition  time  was  4.3  min.  The  sequence  gave  excellent 
image  contrast  for  the  prostate  and  its  surroundings 


3.2  Imaging  Experiments 

We  acquired  high-resolution  MRI  volumes  from  three  volunteers.  For  each  volunteer, 
three  image  volumes  were  obtained  with  an  imaging  session.  Each  volume  was 
acquired  with  compatible  conditions.  Volunteers  laid  supine  with  legs  flat  similar  to 
the  position  in  routine  MR  scanning.  Between  volume  acquisitions,  volunteers  got  off 
the  MRI  table,  stretched,  and  walked  around  to  ensure  that  they  would  assume  a 
different  position  when  they  laid  back  on  the  table.  The  coil  array  was  centered  on  the 
prostate.  All  images  of  a  volunteer  were  acquired  with  the  same  MRI  acquisition 
parameters.  In  total,  there  are  9  pairs  of  high-resolution  MRI  volumes  for  registration. 


3.3  Registration  Experiments 


We  used  9  pairs  of  high-resolution  MR  volumes  to  perform  registration  experiments. 
For  each  volume  pair,  we  extracted  data  from  one  volume  to  simulate  thick  iMRI 
image  slices;  and  then  we  registered  the  simulated  image  slices  to  the  other  volume. 
We  desire  an  iMRI  slice  image  acquisition  method  that  gives  robust,  accurate 
registrations  and  is  relatively  insensitive  to  acquisition  parameters.  Hence,  we 
performed  experiments  to  determine  the  dependence  on  slice  orientation  (transverse, 
sagittal  and  coronal),  on  slice  position  relative  to  the  prostate  (above,  centered,  and 
below)  and  on  image  noise  from  fast  imaging  techniques. 

We  also  performed  SV  registration  experiments  using  the  actual  iMRI  images.  We 
registered  actual  iMRI  image  slices  with  high-resolution  (1.5  T  system)  MR  volumes 
and  visually  evaluated  results.  For  each  volunteer  S1-S3,  there  were  three  high- 
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resolution  MR  volumes  and  30  iMRI  image  slices  giving  90  SV  registration 
experiments,  and  a  total  of  360  experiments. 


4  Results 


4.1  Slice  Orientation  and  Position 

We  determined  SV  registration  results  for  slices  near  the  prostate  in  the  three 
standard  orthogonal  orientations.  Comparing  to  W,  mean  and  standard  deviation 
registration  errors  across  9  volume  pairs  and  45  SV  registration  experiments  were 
0.4  mm  ±  0.2  mm,  0.5  mm  ±  0.2  mm,  and  2.6  mm  ±1.6  mm  for  transverse,  coronal 
and  sagittal  slices  covering  the  prostate,  respectively.  Transverse  slices  worked  best 
because  they  contain  many  relatively  rigid  anatomical  structures.  Coronal  slices 
worked  next  best.  Sagittal  slices  gave  the  largest  error  because  they  contained  a  large 
portion  of  the  deformable  bladder  and  rectum.  We  further  found  that  transverse  slices 
centered  on  the  prostate  produced  better  results  than  those  above  or  below  the 
prostate.  Image  slices  above  included  the  deformable  bladder  that  could  give  an 
inconsistent  structure  from  one  volume  to  the  next.  Image  slices  below  the  prostate 
mainly  contained  muscle  and  fatty  regions  from  the  hips  that  could  deform,  again 
giving  inconsistent  image  data. 


4.2  Noise  Level 

The  typical  iMRI  SNR  under  clinical  conditions  is  about  25.  Even  when  noise  much 
exceeded  this  normal  situation,  registration  results  were  quite  good.  A  100%  success 
rate  was  achieved  with  an  acceptance  criterion  of  <  2.0  mm  even  when  SNR  was  as 
bad  as  10. 


4.3  Actual  iMRI  Images 

As  for  SV  registration  of  actual  iMRI  image  slices  with  a  high-resolution  MR  volume, 
the  contours  overlap  and  overlay  images  show  that  the  prostate  matches  very  well 
(Fig.  3).  Other  visual  inspection  techniques  also  demonstrate  excellent  registration. 
Note  that  a  single  iMRI  image  was  used  to  produce  this  registration  result. 


4.4  Robustness 

The  registration  algorithm  was  quite  robust  for  transverse  slices  covering  the  prostate. 
Using  the  nine  volume  pairs  from  three  volunteers,  the  algorithm  never  failed  for  any 
transverse  slice  covering  the  prostate.  The  time  for  an  SV  registration  was  typically 
about  5  sec  on  a  Pentium  IV,  1.8  GHz  CPU,  with  1  Gbytes  of  memory.  The  algorithm 
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was  written  in  IDL  (Interactive  Data  Language,  Research  System  Inc.,  CO)  and  could 
probably  be  made  much  faster  in  a  lower  level  language  such  as  C. 


Fig.  3  Prostate  MR  images.  The  top  is  the  high  resolution  MR  image.  The  bottom  is  the  actual 
iMRI  image;  the  rectangular  window  at  the  center  is  the  transparency  display  of  both  images. 
The  prostate  boundary  was  segmented  from  high  resolution  MR  image  that  matches  well  with 
the  prostate  in  iMRI  image.  Images  are  from  S3. 


4.5  Comparison  of  Different  Interpolation  Methods 

Computation  time  and  registration  accuracy  are  two  main  factors  to  consider  when 
choosing  interpolation  methods.  Using  tri-linear  interpolation,  the  time  for  an  S V 
registration  was  typically  about  5  sec  on  a  Pentium  IV,  1.8  GHz  CPU,  with  1Gbyte  of 
memory.  If  the  re-normalized  sine  interpolation  method  was  used,  the  time  was  "10 
min,  a  duration  not  acceptable  for  our  application.  The  algorithm  was  written  in  IDL 
and  could  probably  be  made  faster  in  a  lower  level  language  such  as  C.  We  did  not 
use  nearest  neighbor  because  of  the  obvious  insufficient  accuracy  as  induced  from  its 
flat  peak  of  the  similarity  surfaces  in  Figure  2.  A  call  to  the  Simplex  optimization 
typically  resulted  in  50  to  105  similarity  evaluations  before  the  tolerance  value 
(0.001)  was  reached. 


28 


9 


Lecture  Note  of  Computer  Science,  MICCAI 2003 


5  Discussions  and  Conclusion 

The  registration  experiments  presented  here  provided  fairly  comprehensive  tests  for 
the  potential  application  in  iMRI-guided  RF  thermal  ablation  of  the  prostate. 
Simulation  provided  an  efficient  way  to  extensively  evaluate  registration 
performance.  The  algorithm  was  extremely  robust  to  noise  levels,  far  beyond  those 
encountered  in  clinical  iMRI  applications.  The  actual  iMRI  images  acquired  under 
more  realistic  conditions  further  tested  practicality.  Images  from  the  iMRI  system 
contained  more  noise  and  had  less  contrast  than  those  from  the  1.5  T  scanner. 
Registration  quality  was  comparable  to  that  of  simulation  experiments.  Registration 
time  can  probably  be  improved  considerably  using  optimized  C  code  rather  than  IDL. 
If  registration  is  done  in  the  background  in  a  seamless  way,  the  time  for  registration  is 
probably  quite  acceptable. 

We  recommend  that  image  data  are  obtained  under  comparable  conditions  by 
keeping  a  similar  posture  and  by  taking  clinical  measures  to  reduce  rectal  and  bladder 
filling.  In  certain  lesions,  such  as  cysts  or  necrotic  tumor,  persistent  deformation  is 
possible;  however,  we  can  see  such  deformations  in  the  live-time  interventional  MRI 
images  and  very  probably  mentally  correct  the  registered,  fused  images.  We 
previously  reported  a  warping  registration  method  17  that  can  correct  deformations  at 
the  expense  of  additional  complexity,  time,  and  possibly  robustness. 

Finally,  we  conclude  that  the  slice  to  volume  registration  algorithm  is  quite  robust 
for  transverse  slice  images  covering  the  prostate  and  that  registration  accuracy  is 
probably  sufficiently  accurate  to  aid  iMRI-guided  thermal  ablation  of  prostate  cancer. 
It  is  quite  feasible  to  include  previously  acquired  high-resolution  MRI  and  nuclear 
images  into  iMRI-guided  treatment  procedures.  We  are  beginning  to  explore  this 
application  in  animal  experiments. 
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ABSTRACT 

We  created  and  evaluated  an  almost  fully  automated,  three-dimensional  warping 
registration  algorithm  using  mutual  information  and  a  thin  plate  spline  (TPS) 
transformation  for  MR  images  of  the  prostate  and  pelvis.  In  the  first  step,  an  automatic 
rigid  body  registration  was  used  to  capture  the  global  transformation.  Algorithm  features 
included  a  multi-resolution  approach,  two  similarity  measures,  and  automatic  restarting  to 
avoid  local  minimums.  In  the  second  step,  local  feature  points  were  registered.  An 
operator  entered  only  five  feature  points  (FP’s)  located  at  the  prostate  center,  the  left  and 
right  hip  joints,  and  die  left  and  right  distal  femurs.  The  program  used  the  prostate  center 
and  radial  projections  to  automatically  detect  FP’s  at  the  external  pelvic  skin  surface. 
FP’s  were  also  automatically  determined  along  the  femurs.  The  program  optimized 
mutual  information  of  a  small  cubic  volume  of  interest  (VOI)  centered  on  each  FP  by 
varying  three  translations  and  three  rotations.  For  each  optimized  VOI,  nine  control  points 
(CP’s)  were  obtained  at  the  center  and  near  the  comers  of  a  small  cube  circumscribed  by 
the  VOI.  More  than  600  CP’s  were  used  to  establish  a  TPS  transformation  for  the 
deformation  of  the  pelvic  region  and  the  prostate.  Over  100  experiments  from  healthy 
volunteers  determined  the  quality  of  registration  under  conditions  found  in  potential 
applications  in  prostate  cancer  treatment  and  interventional  MRI  guided  therapies.  Images 
were  acquired  in  the  diagnostic  (supine)  and  treatment  position  (supine  with  legs  raised). 
For  10  volume  pairs  obtained  in  much  different  positions,  warping  registration  was  much 
better  than  rigid  body  registration.  Using  various  visualization  techniques,  it  was  clear 
that  warping  rectified  the  sometimes  significant  pelvic  misalignment  by  the  rigid  body 
method.  Gray  value  measures  of  registration  quality  including  mutual  information, 
correlation  coefficient,  and  intensity  difference  all  improved  with  warping.  Importantly, 
the  prostate  tended  to  be  more  accurately  registered  with  warping;  the  distance  between 
prostate  volume  centroids  was  only  0.7  ±  0.2  mm  following  warping  as  compared  to 
4.9  ±3.4  mm  with  rigid  body  registration.  From  visual  and  quantitative  measurements, 
we  found  that  our  almost  fully  automatic  method  worked  as  well  as  a  previously  reported 
method  using  warping  registration  with  manually  placed  control  points.  The  method  was 
computationally  efficient  for  the  600  control  points  typically  used.  We  believe  that  the 
warping  registration  method  will  provide  a  powerful  tool  for  many  applications  in  the 
pelvic  and  prostate  imaging. 


Key  words:  automatic  warping  image  registration,  mutual  information,  thin  plate  spline, 
interventional  magnetic  resonance  imaging  (iMRI),  prostate  cancer. 
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I.  INTRODUCTION 

We  are  investigating  three-dimensional  (3D)  warping  image  registration  to  be  used  in 
applications  of  prostate  cancer  diagnosis,  staging,  and  therapy.  In  particular,  we  are 
interested  in  applications  related  to  the  minimally  invasive  interventional  MRI  (iMRI) 
guided  treatment  of  prostate  cancer.  At  our  institution,  we  currently  use  iMRI  on  a  low- 
field  open  magnet  system  to  guide  radiofrequency  (RF)  thermal  ablation  of  abdominal 
cancer[l]-[3],  and  we  are  investigating  this  method  for  prostate  cancer  treatment.  A 
unique  feature  of  iMRI-guided  thermal  ablation  is  that  therapy  can  be  monitored  with  MR 
either  by  acquiring  images  of  thermally  induced  lesions  or  by  measuring  temperature.  In 
addition,  MR  imaging  of  the  prostate  is  desirable  because  it  more  accurately  delineates 
the  prostate  than  does  CT  [4][5],  which  can  overestimate  prostate  volume  [6][7],  and 
ultrasound,  which  has  a  tendency  to  underestimate  the  extent  of  lesions  [8]. 

Several  applications  in  prostate  imaging  require  registration.  First,  comparison  of 
registered  MR  images  acquired  before  and  immediately  after  RF  ablation  can  be  used  to 
determine  whether  a  tumor  is  adequately  treated.  This  is  particularly  helpful  in  instances 
where  the  edematous  response  to  treatment  can  be  confused  with  a  highly  perfused  tumor. 
Second,  other  treatment  methods  such  as  radiation  therapy,  brachytherapy,  and  surgery, 
will  also  be  aided  by  registration  of  images  from  pre-,  intra-,  and  post-therapy  for 
treatment  planning,  guidance,  and  assessment.  Third,  inter-modality  registration  will  be 
useful  because  there  are  emerging  techniques  in  nuclear  medicine  [9]  and  MR 
spectroscopy  [10]  that  promise  to  detect  and  delineate  prostate  tumor.  We  have  an  active 
project  aimed  at  registering  and  fusing  SPECT  monoclonal  antibody  images  with  MR  to 


33 


4 


aid  interventional  MRI  guided  treatments.  Fourth,  registration  of  serial  examinations  can 
be  used  to  follow  regression/progression  of  tumors. 

There  are  challenges  to  the  pelvis  and  prostate  registration.  First,  pelvic  regions  can 
change  shape  significantly,  unlike  the  brain  to  which  registration  has  been  most  often 
applied.  Different  patient  positions  such  as  legs  up  and  down  can  cause  the  movement 
and  deformation  of  internal  organs.  Second,  the  normal  prostate  is  a  small  organ  that 
when  healthy  measures  only  about  3.8  cm  in  its  widest  dimension  transversely  across  the 
base  [11].  Third,  the  small  prostate  is  located  below  a  much  larger  bladder  that  can 
change  shape  and  size.  The  prostate  might  move  relative  to  the  pelvic  bones  due  to 
changes  in  bladder  and  rectal  filling  [12][13].  The  alignment  of  the  pelvic  bones,  a  most 
prominent  anatomical  feature  in  MR  gray-scale  images,  does  not  necessarily  mean  that 
the  prostate  is  aligned. 

Some  reports  describe  methods  for  rigid  body  registration  in  the  pelvis  or  prostate 
[12][14]-[21],  Some  of  these  methods  require  either  segmentation  or  visual  identification 
of  structures.  For  example,  manual  registration  has  been  used  where  an  operator  cues  on 
segmented  vascular  structures  [22]  or  other  anatomical  landmarks  [14]-[16].  Others  have 
used  automated  3D  schemes  that  match  contours  of  bones  and  sometimes  other  structures 
that  are  extracted  using  manual  or  interactive  segmentation  [18][19][23].  Manual 
segmentation  has  also  been  used  to  create  surfaces  for  automatic  registration  [20] [21]. 

We  previously  described  a  gray-scale,  mutual  information,  rigid  body  volume-to- 
volume  (W)  registration  method  for  the  pelvic  and  prostate  MR  images  [24],  For  volume 
pairs  acquired  over  a  short  time  span  with  the  volunteer  in  a  similar  position,  rigid  body 
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registration  accuracy  of  both  the  prostate  centroid  (typically  <  1  mm)  and  bony  landmarks 
(average  1.6  mm)  was  on  the  order  of  a  voxel  (~  1.4  mm).  The  centroid  error  was  slightly 
smaller  because  the  prostate  was  at  the  volume  center  and  rotation  errors  had  less  effect 
on  it.  The  localization  error  in  finding  3D  points  from  bony  landmarks  is  probably  greater 
than  that  of  finding  centroids  of  relatively  large  prostate  volumes  where  segmentation 
errors  average  out.  With  rigid  body  registration,  we  obtained  larger  prostate  centroid 
displacements  (2.8  to  10.0  mm)  when  acquisitions  were  obtained  tinder  much  different 
conditions  (e.g.,  legs  flat  and  legs  raised)  giving  large  anatomical  deformations.  Rigid 
body  registration  of  the  pelvis  is  inadequate  under  such  conditions  [24], 

Warping  registration  is  a  solution,  and  there  are  a  number  of  relevant  reports  [25]- 
[33].  However,  these  applications  were  mainly  for  brain  [29][31][33]-[35]  and  breast 
[27][30][36];  far  fewer  applications  have  been  described  for  the  abdomen  [28][32][37]. 
As  for  the  prostate,  a  finite  element  model  was  used  to  deform  the  gland  during 
brachytherapy  treatments  and  this  method  also  requires  manual  segmentation  of  the  gland 
[38]. 

We  recently  reported  a  warping  registration  method  that  used  many  manually  selected 
control  points  [39][40].  After  automatic  global  rigid  body  registration,  the  operator 
manually  selected  over  180  control  points  at  the  prostate  center,  pelvic  surface  and 
internal  structures.  The  program  automatically  optimized  each  control  point  location  by 
displacing  it  in  the  x,  y,  and  z  directions  with  respect  to  the  reference  volume  until  the 
mutual  information  computed  over  a  small  cube  of  voxels  was  maximized.  The  thin  plate 
spline  (TPS)  transformation  was  then  applied  to  express  the  deformation  of  the  pelvic 
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region  and  the  prostate.  This  interactive  method  was  applied  to  pelvic  MR  images  and 
lung  CT-PET  images  [41].  The  time  required  for  control  point  selection  was  a  limitation. 

In  this  study,  we  build  upon  our  previous  experience  and  develop  an  almost  fully 
automatic  warping  registration  method.  Our  goal  is  to  automate  the  algorithm  to  save 
time  and  labor  without  losing  registration  quality.  In  registration  experiments,  we  use 
high-resolution  MR  image  volumes  of  the  pelvis  and  prostate  in  order  to  determine  any 
uncompensated  deformation  following  registration.  We  use  image  data  that  shows 
considerable  deformation;  e.  g.,  images  acquired  in  the  diagnostic  (supine)  and  treatment 
positions  (supine  with  legs  raised).  We  qualitatively  and  quantitatively  compare  results  of 
the  new  warping  algorithm  to  the  previous,  more  manual  version,  and  to  rigid  body 
registration. 


II.  REGISTRATION  ALGORITHM 
A.  Similarity  Measurements 

We  used  two  similarity  measures,  mutual  information  (MI)  and  correlation  coefficient 
(CC)  to  perform  a  gray-scale  registration.  Suppose  one  volume  R  is  the  reference,  and  the 
other  F  is  floating.  Their  mutual  information  MI(R,  F)  is  given  below  [42]. 

r.f  pR(r)-pF(f ) 
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The  joint  probability  pRF (r,f) and  the  marginal  probabilities  pR(r) of  the  reference 
image  and  pF(f)  of  the  floating  image,  can  be  estimated  from  the  normalized  joint 
intensity  histogram.  The  correlation  coefficient  CC(R,F)  is  given  below  [43]. 

CC(R  F)  im-ww-TO 

Here  R(r),  F(f)  denote  the  average  intensities  of  the  reference  and  floating  volumes  and 
the  summation  includes  all  voxels  within  the  overlap  of  both  volumes. 


B.  Global  Rigid  Body  Registration  Algorithm  with  Special  Features 

Rigid  body  registration  is  the  first  step,  and  it  captures  the  global  transformation  of 
two  volumes.  We  previously  reported  a  similar  method  includes  special  features  to 
improve  robustness  for  MR  pelvic  images  [24],  We  use  a  multi-resolution  approach  and 
perform  registration  from  low  to  high  resolution.  At  low  resolution,  we  resample  both 
images  at  1/4  or  1/2  number  of  voxels  along  each  linear  dimension,  respectively.  The 
similarity  measure  is  iteratively  optimized  by  varying  the  six  rigid  body  transformation 
parameters  (three  translations  and  three  angles).  We  use  CC  at  the  two  lower  resolutions 
because  it  gives  fewer  local  maximums  [44][45],  and  because  it  can  be  calculated  faster 
than  MI.  We  use  MI  at  full  resolution  because  the  peaked  similarity  function  gives  a  more 
precise  solution  than  CC  [44].  To  avoid  local  maximums,  we  include  a  restarting  feature 
where  registration  is  restarted  with  randomly  perturbed  parameters  obtained  from  a 
uniform  distribution  about  the  initial  transformation  values  at  the  current  resolution.  The 
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algorithm  restarts  until  the  absolute  CC  is  above  a  threshold  of  0.5  or  the  maximum 
number  of  restarts  is  reached.  Absolute  CC  is  used  for  the  restart  test  rather  than  MI 
because  CC  has  a  well-defined  range  between  0  and  1,  and  because  CC  has  fewer 
problems  with  local  and  incorrect  global  maximums  for  registrations  at  low  resolution  far 
from  the  optimum  value  [24]  [46]. 

We  record  all  important  results  following  an  optimization  cycle  including  the  CC 
and/or  MI  values,  the  number  of  restarts,  and  the  transformation  parameters.  At  the  end  of 
processing  at  a  lower  resolution,  we  always  select  the  transformation  parameters  having 
the  maximum  CC  value.  We  then  scale  the  translation  parameters  appropriately  and 
assign  the  new  parameters  to  be  initial  values  at  the  next  higher  resolution.  At  the  highest 
resolution,  we  select  the  final  transformation  parameters  to  be  those  with  the  maximum 
MU  value. 

Typical  parameter  values  are  now  described.  We  use  an  initial  guess  at  the  lowest 
resolution  of  all  zeros  because  the  patient  is  normally  oriented  approximately  the  same 
way  from  one  scan  to  the  next.  We  set  the  maximum  numbers  of  restarts  at  10,  5,  and  3, 
from  low  to  high  resolution,  respectively. 


C.  Warping  Registration  by  Optimizing  Control  Points 

Following  the  global  rigid  body  transformation,  we  perform  local  warping  registration 
to  rectify  the  misalignment  of  the  pelvic  region  and  the  prostate.  Below  we  describe 
details  of  the  various  steps  of  the  algorithm  outlined  in  Figure  1. 
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C.l.  Semi-Automatic  Detection  of  Feature  Points 

After  global  transformation,  we  manually  selected  five  corresponding  pairs  of  feature 
points  (Figure  2)  from  the  two  volumes.  Locations  are:  the  prostate  center,  the  two  hip 
joints,  and  the  two  distal  femurs.  These  feature  points  are  especially  useful  when 
registering  volumes  obtained  in  the  diagnostic  (supine)  and  treatment  (supine  with  legs 
raised)  positions  [24][39].  Transverse  slices  are  used  to  select  these  points  because  they 
better  show  displacement  of  the  legs.  Images  from  both  volumes  are  displayed  side  by 
side  to  aid  interpretation. 

Using  these  five  FP’s,  the  program  creates  all  other  FP’s  automatically.  First,  for  each 
volume,  a  line  segment  is  created  between  the  FP’s  at  the  hip  joint  and  the  distal  femur 
(dashed  lines  in  Figure  2).  Each  segment  is  divided  into  8  equal  lengths  to  obtain  7 
additional  FP’s.  Altogether,  we  obtained  9  corresponding  FP  pairs  for  each  femur. 

Second,  the  prostate  center  is  used  to  create  FP’s  at  the  pelvic  surfaces  (Figure  3a). 
Many  radial  lines  are  drawn  through  the  prostate  center  in  a  two-dimensional  (2D) 
transverse  slice.  The  angle  between  lines,  the  angle  increment,  was  normally  1 8,  20,  or  30 
degrees.  Note  that  we  exclude  the  vertical  line  because  it  normally  intersects  the  buttock 
groove  and/or  the  penis,  regions  of  relatively  little  interest  that  can  change  considerably 
from  one  acquisition  to  the  next.  The  gray-scale  intensity  of  the  pixels  along  each  line  is 
obtained  (Figure  3b).  The  signal  in  air  is  noise  and  normally  less  than  15  gray  levels  as 
compared  to  a  mean  signal  of  about  90  and  maximum  signal  of  about  500  in  the  tissues. 
The  pelvic  skin  surface  gives  high  intensity  signals,  above  30  gray  levels,  because  of  the 
fat  beneath  the  skin.  We  searched  each  line  for  the  most  distant  point  above  a  threshold,  T 
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(Figure  3b)  and  set  this  position  to  be  a  surface  feature  point  (Figure  3a).  We  normally 
choose  a  threshold  of  20  gray  levels.  After  processing  all  lines  in  an  image  slice,  the 
program  repeats  the  procedure  for  4  other  transverse  slices  that  are  selected  with  a  gap  of 
about  10-mm  away  from  each  other.  These  transverse  slices  span  ~  40  mm,  much  of  a 
typical  prostate. 

For  a  typical  registration  with  an  angle  increment  of  18  degrees,  there  are  91  pairs  of 
FP’s:  one  at  the  prostate  center,  18  at  the  femurs,  and  72  at  the  exterior  pelvic  surface. 
Their  3D  coordinates  are  stored.  Because  of  the  subsequent  optimization,  the  FP  pairs  do 
not  have  to  exactly  lie  on  corresponding  features;  an  error  of  up  to  15  mm  or  «  10  voxels 
is  well  tolerated. 

C.2.  Optimization  of  Corresponding  Feature  Points 

Corresponding  feature  points  are  optimized  using  a  mutual  information  similarity 
measure.  A  small  cubic  volume  of  interest  (VOI)  is  centered  at  each  FP.  It  can  be  32, 48 
or  64  voxels  on  a  side,  and  as  reported  later,  the  selection  of  VOI  size  depends  on  the 
amount  of  warping  required.  To  perform  the  optimization,  the  floating  VOI  is  translated 
and  rotated  with  respect  to  the  reference  VOI  and  mutual  information  is  computed  over 
corresponding  voxels.  For  each  floating  VOI,  there  are  six  transformation  parameters  to 
optimize:  three  displacements  in  x,  y,  and  z,  and  three  angles  about  each  of  these  three 
axes.  A  Simplex  algorithm  is  used  for  optimization  [47].  Optimal  transformation 
parameters  are  independently  obtained  and  recorded  for  each  of  the  floating  feature 
points.  On  rare  occasion,  one  or  more  of  the  translation  or  rotation  parameters  of  a 
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floating  FP  are  larger  than  a  reasonable  value  (e.g.,  40  nun  or  45  degrees),  and  we 
eliminate  it. 

C.3.  Determination  of  Control  Points 

For  each  optimized  feature  point  VOI,  we  create  nine  control  points  (CP’s).  As  shown 
in  Figure  4,  there  is  a  smaller  cube  about  half  the  dimensions  of  a  bigger  cube  that  is  used 
to  compute  MI.  The  nine  CP’s  are  the  eight  comers  and  the  center  of  the  small  cube. 
These  CP’s  encode  the  position  and  orientation  of  the  VOI.  For  feature  points  near  the 
MR  image  volume  border,  some  CP’s  may  not  be  in  the  volume,  and  we  exclude  them. 
The  total  number  of  CP's  is  usually  >  600  for  an  angle  increment  of  1 8  degrees. 

C.4.  TPS  Transformation  Using  Optimal  Points 

The  final  major  step  is  to  warp  the  floating  volume  using  the  optimal  CP  coordinates 
to  establish  a  three-dimensional  TPS  transformation  [27][48]-[50].  Tri-linear 
interpolation  is  used. 


D.  Additional  Details 

There  are  several  preprocessing  details.  The  input  MR  volume  is  a  3D  MR  acquisition 
giving  256  x  256  x  128  nearly  isotropic  voxels  over  a  field  of  view  covering  the  whole 
pelvis.  We  create  isotropic  voxels  of  about  1.4  mm  on  a  side  using  tri-linear  interpolation. 
Before  optimization,  we  optionally  bin  the  gray  scale  values  of  two  volumes  to  256;  this 
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procedure  helps  the  algorithm  perform  better  for  multi-modality  registrations  such  as 
CT/PET  where  their  gray  scales  are  tremendously  different  [41].  For  both  rigid  body 
registration  and  VOI  optimization,  we  use  tri-linear  interpolation  to  obtain  samples  at 
real-valued  voxel  locations.  Optimization  of  similarity  ends  either  when  the  maximum 
number  of  calculations  is  reached  (typically  500)  or  the  fractional  change  in  the  similarity 
function  is  smaller  than  a  tolerance  (typically  0.001).  The  program  is  written  in  IDL 
(Interactive  Data  Language,  Research  System  Inc.,  Boulder,  CO). 


m.  EXPERIMENTAL  METHODS 

A.  Image  Acquisition 

All  MRI  volumes  were  acquired  using  a  1.5  T  Siemens  MRI  system  (Magnetom 
Symphony,  Siemens  Medical  Systems,  Erlangen,  Germany).  An  8-element  phased  array 
body  coil  was  used  to  ensure  coverage  of  the  prostate  with  a  uniform  sensitivity. 
Typically  two  anterior  and  two  posterior  elements  were  enabled  for  signal  acquisition. 
We  used  two  different  MR  sequences.  First,  we  used  a  3D  RF  spoiled  gradient  echo 
steady  state  pulse  sequence  (FLASH)  with  TR/TE/flip  parameters  of  12/5.0/60  which 
give  256x256x128  voxels  over  a  330x330x256-mm  field  of  view  (FOV)  to  yield 
1.3xl.3x2.0-mm  voxels  oriented  to  give  the  highest  resolution  for  transverse  slices.  The 
acquisition  time  was  5.6  min.  This  sequence  was  good  for  pelvic  imaging  but  was  not 
ideal  for  the  prostate.  It  was  used  for  volunteer  SI.  Second,  we  used  a  3D  rapid  gradient 
echo  sequence  (PSIF)  designed  to  acquire  the  spin-echo  component  of  the  steady  state 
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response.  The  spin  echo  component  formed  immediately  prior  to  the  RF  pulse,  and  it  was 
shifted  toward  the  prior  RF  pulse  through  appropriate  gradient  waveform  design.  The 
sequence  with  9.4/5.0/60  (TR/TE/flip)  yielded  160x256x128  voxels  over  a  219x350x192- 
mm  rectangular  FOV  and  1.4xl.4xl.5-mm  voxels  oriented  to  give  the  highest  resolution 
for  transverse  slices.  There  was  over  sampling  at  31%  in  the  slice  direction  to  reduce 
aliasing  artifacts.  The  acquisition  time  was  4.3  min.  Most  often,  we  used  the  second 
sequence,  which  gave  excellent  image  contrast  for  the  prostate  and  its  surroundings.  It 
was  used  for  volunteers  S2  and  S3. 


B.  Imaging  Experiments 

We  acquired  3D  MRI  volume  images  from  three  normal  volunteers  under  a  variety  of 
conditions  simulating  anticipated  conditions  in  diagnostic  and  treatment  applications.  In 
the  diagnostic  position,  the  volunteer  laid  supine  throughout  MR  scanning.  In  the 
treatment  position,  the  volunteer  was  supine,  and  his  legs  were  supported  at  30°-60° 
relative  to  the  horizon  and  separated  in  a  “V”  with  an  angle  of  60°-90°  between  two  legs. 
This  is  similar  to  the  lithotomy  position  used  in  prostate  therapies,  and  it  should  provide 
access  for  needle  insertion  in  brachytherapy  or  RF  thermal  ablation.  For  each  volunteer, 
image  volumes  were  typically  obtained  on  the  same  day  within  an  imaging  session. 
Between  volume  acquisitions,  volunteers  got  off  the  MRI  table,  stretched,  and  walked 
around  to  ensure  that  they  would  assume  a  different  position  when  they  laid  back  on  the 
table.  The  coil  array  was  centered  on  the  prostate.  All  images  of  a  volunteer  were 
acquired  with  the  same  MRI  acquisition  parameters  so  as  to  ensure  very  similar  gray 


43 


14 


values.  In  total,  there  are  4,  4,  and  5  volumes  for  each  volunteer,  SI,  S2,  and  S3, 
respectively.  For  each  volunteer,  one  volume  was  in  the  treatment  position,  the  other  were 
in  the  diagnostic  position. 

We  performed  registration  experiments  using  treatment-diagnosis  volume  pairs.  For 
volunteers  SI,  S2,  and  S3,  there  were  3,  3,  and  4  treatment-diagnosis  volume  pairs, 
respectively.  Rigid  body  and  warping  registration  were  applied  to  each  pair.  Additionally, 
we  tested  the  methods  on  10  volume  pairs  obtained  in  the  same  diagnostic  position. 


C.  Registration  Evaluation 

We  used  multiple  visualization  features  of  RegViz,  a  program  written  in  IDL  and 
created  in  our  laboratory  for  visualizing  and  analyzing  registered  image  volumes.  First, 
color  overlay  displays  were  used  to  evaluate  overlap  of  structures.  One  image  was 
rendered  in  gray  and  the  other  in  the  “hot  -  iron”  color  scheme  available  in  IDL.  To 
visualize  potential  differences,  it  was  quite  useful  to  interactively  change  the  contribution 
of  each  image  using  the  transparency  scale.  Second,  we  used  a  sector  display,  which 
divided  the  reference  and  registered  images  into  rectangular  sectors  and  created  an  output 
image  by  alternating  sectors  from  the  two  input  images.  Even  subtle  shifts  of  edges  could 
be  clearly  seen.  Third,  we  manually  segmented  prostate  boundaries  in  image  slices  and 
copied  them  to  corresponding  slices  from  the  other  volume.  This  enabled  visual 
determination  of  the  overlap  of  prostate  boundaries  over  the  entire  volume.  We  applied 
the  same  method  to  evaluate  pelvic  registration. 
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Correlation  coefficient  and  mutual  information  were  calculated  as  indicators  of 
regitration  quality.  Since  volumes  to  be  registered  were  acquired  using  the  same 
acquisition  parameters,  high  absolute  CC  values  were  obtained  when  registration  was 
good  [30].  The  higher  the  MI  values,  the  better  the  two  volumes  are  registered. 

We  compared  the  new  semi-automatic  method  with  a  warping  registration  that  used 
manually  selected  control  points  as  reported  by  us  [39].  Briefly,  in  the  previous  method, 
we  performed  a  rigid  body  registration,  manually  selected  about  180  control  points, 
optimized  their  locations,  and  warped  the  volume  using  TPS  interpolation. 

Finally,  we  measured  potential  displacements  of  the  3D  centroid  of  the  prostate  to 
assess  prostate  registration  error.  We  used  RegViz  to  manually  segment  the  prostate 
across  all  image  slices  and  calculated  the  3D  centroid. 


IV.  RESULTS 

A.  Determination  of  Feature  Points 

The  method  for  finding  corresponding  FP’s  was  quite  successful.  After  rigid  body 
registration,  our  visualization  tool  made  it  easy  to  approximately  locate  the  prostate 
center,  hip  joints,  and  the  femurs.  The  automatic  detection  method  for  identifying  the 
external  pelvic  surface  was  quite  reliable.  For  all  volume  pairs,  the  program  correctly 
detected  all  surface  points  using  all  angle  increments  and  a  threshold  of  20  gray  levels. 
Figure  5  shows  surface  FP’s  for  a  typical  treatment-diagnostic  volume  pair.  The  method 
for  finding  FP’s  along  the  femurs  also  worked  in  a  robust  fashion. 
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In  over  60  warping  registration  experiments  using  10  treatment-diagnosis  volume 
pairs,  we  found  that  more  FP’s  generally  improved  registration  quality.  Figure  6  plots  MI 
values  of  registered  volumes  after  warping  as  a  function  of  angle  increment.  When  the 
angle  increment  is  less  than  20  degrees,  MI  saturates.  As  the  angle  increment  increases, 
less  FP’s  are  used  and  the  MI  value  decreases.  The  downside  of  adding  more  CP's  is  the 
increased  time  for  registration.  We  use  an  angle  increment  of  18  degrees  in  subsequent 
experiments. 

Adding  feature  points  along  the  femurs  was  important  for  good  registration  of  internal 
structures.  When  only  surface  FP's  were  used,  the  femurs  and  its  surrounding  structures 
did  not  register  well  (not  shown)  even  with  a  very  large  number  of  surface  points  as 
obtained  with  an  angle  increment  of  9  degrees. 

B.  Registration  Quality  of  Warping  versus  Rigid  Body  Registration 

In  Figure  7,  we  compare  warping  and  rigid  body  registration  for  a  typical  volume  pair 
from  the  treatment  and  diagnostic  positions.  Following  warping  registration,  the  prostate 
boundary  overlap  is  excellent  (Figure  7e)  and  within  manual  segmentation  error  as 
assessed  by  an  experienced  operator.  Using  rigid  body  registration,  there  is  significant 
misalignment  throughout  large  regions  in  the  pelvis  as  shown  in  the  overlap  image 
(Figure  7f),  and  alignment  is  greatly  improved  with  warping  (Figure  7g)  where  the  pelvis 
matched  very  well  even  at  the  outer  surfaces.  Similar  results  were  obtained  in  transverse 
slices  throughout  the  prostate.  Other  visualization  methods  such  as  two-color  overlays 
and  difference  images  also  show  excellent  matching  of  structures. 
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Visual  inspections  were  further  verified  by  quantitative  measurements.  Figure  8  shows 
the  prostate  centroid  displacement  following  rigid  body  and  warping  registration. 
Warping  significantly  decreased  the  displacement,  indicating  much  better  registration  of 
the  prostate.  Following  warping,  the  mean  displacement  was  only  0.7  ±  0.2  mm  cross  all 
volume  pairs.  The  later  number  is  the  standard  deviation.  However,  the  mean  was 
4.9  ±3.4  mm  for  rigid  body  registration. 

A  typical  example  is  the  second  pair  of  volunteer  S2.  Following  rigid  body 
registration,  the  prostate  was  misaligned  with  a  displacement  to  the  posterior  of  «  13.6 
mm  when  in  the  treatment  position  (Figure  7d).  A  prostate  rotation  of  ~  14  degree  along 
left-right  axis  was  also  found  following  rigid  body  registration.  Following  warping 
registration,  the  3D  prostates  matched  very  well  and  the  centroid  calculated  from 
segmented  images  displaced  by  only  0.9  mm,  or  0.7  voxels. 

Additionally,  we  examined  10  volume  pairs  with  both  volumes  acquired  in  the 
diagnostic  position.  In  all  such  cases,  rigid  body  registration  worked  as  well  as  warping 
registration.  There  were  no  noticeable  deformations  in  the  pelvic  region,  and  prostate 
centroids  typically  displaced  less  than  1.0  mm  between  the  two  registered  volumes.  Note 
that  this  was  obtained  even  though  volunteers  always  got  up  from  the  table  and  moved 
around  before  being  imaged  again.  In  what  follows,  we  only  report  results  of  treatment- 
diagnosis  volume  pairs. 

In  Figure  9,  we  compare  rigid  body,  manual  warping,  and  our  new  almost  fully 
automatic  warping  methods.  Correlation  coefficient  (CC)  values  between  registered 
volumes  of  treatment-diagnosis  are  plotted.  Both  semi-automatic  and  manual  warping 
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significantly  increased  CC  values  in  every  case,  and  a  paired  two-tailed  t  test  indicated  a 
significant  effect  of  warping  at  p<0.5%.  The  semi-automatic  warping  registration 
worked  better  than  the  manual  one  in  7  out  of  10  cases.  This  is  probably  because  we  used 
more  (~  600)  CP’s  with  the  semi-automated  method  than  the  manual  method  (~  180). 
Recall  that  with  the  manual  method  each  FP  was  optimized  for  translation  and  not 
rotation  and  each  FP  gave  one  CP 


C.  Algorithmic  Implementation 

The  rigid  body  registration  is  robust  as  previously  reported  by  us  [24].  For  10  volume 
pairs  acquired  in  the  same  diagnostic  position,  registration  was  100%  successful  with 
prostate  centroids  typically  displaced  <  1  mm  following  registration.  For  10  treatment- 
diagnosis  volume  pairs,  the  pelvic  region  was  aligned  except  for  local  deformations  and 
displacements.  The  time  for  rigid  body  registration  was  typically  5  minutes  on  a  Pentium 
IV,  1 .8  GHz  CPU,  with  1 .0  G  Bytes  of  memory. 

We  report  some  details  on  warping  registration  for  a  typical  volume  pair  from 
volunteer  S2.  The  angle  increment  was  18  degrees  and  the  total  number  of  FP's  was  91. 
After  FP  optimization,  21  FP's  were  rejected,  and  70  FP's  remained  that  produced  630 
CP's.  Excluding  17  invalid  CP's  outside  the  volume  region,  there  were  613  useful  CP's  for 
the  TPS  transformation.  Following  rigid  body  registration,  the  mean  distance  between  the 
reference  and  floating  FP's  was  18.6  ±  12.4  mm,  where  the  latter  number  is  the  standard 
deviation.  The  maximum  distance  was  58.2  mm.  The  maximum  rotation  of  a  VOI  was 
40.3  degrees.  Using  the  same  computer  as  described  above,  for  volumes  with 
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256x256x140  voxels,  the  warping  registration  took  about  30  minutes  with  TPS  warping 
occupying  ~  86%  of  the  time.  Very  probably,  the  time  could  be  greatly  improved  with 
optimized  C  code  rather  than  the  high  level  language,  BDL. 

VOI  size  was  an  important  parameter  to  control  the  amount  of  warping.  We 
performed  experiments  to  find  the  optimal  VOI  size  using  sizes  of  16,  32,  64,  and  72 
voxels.  We  quickly  found  that  a  VOI  size  of  16  or  72  voxels  on  a  side  gave  unsatisfactory 
results  because  warping  that  was  either  too  much  or  too  little,  respectively.  A  VOI  size  of 
64  worked  somewhat  better  than  other  VOI  sizes  for  SI  and  S3  because  there  was 
relatively  little  deformation  between  volume  pairs.  However,  a  VOI  size  of  32  voxels  on 
a  side  performed  best  for  volunteer  S2  because  there  was  more  deformation  in  this 
case  (Figure  7). 

The  method  for  obtaining  CP’s  from  FP’s  satisfactorily  encoded  the  local  position  and 
orientation  for  warping.  As  described  previously,  we  used  the  comers  and  the  center  of  a 
small  cubic  volume  contained  within  the  FP  VOI.  Using  the  comers  of  the  FP  VOI  gave 
unsatisfactory  results  from  visual  inspection  as  well  as  CC  and  MI  values. 


V.  DISCUSSION 

A.  Applicability  of  Warping  Registration 

For  MR  images  of  the  pelvis  and  prostate,  warping  registration  is  desirable  whenever 
images  are  acquired  in  different  positions.  Local  deformations  throughout  the  pelvis  can 
be  corrected,  and,  more  importantly,  the  prostate  can  be  accurately  registered.  However, 
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when  images  are  acquired  in  the  same  position  under  comparable  conditions  such  as  the 
case  called  diagnosis-diagnosis,  rigid  body  registration  worked  satisfactorily  [24]. 
Similarly,  if  one  were  to  reproduce  the  treatment  position  reasonably  well,  one  could 
probably  again  get  very  good  results  with  rigid  body  registration. 

The  high-resolution  MR  images  provide  a  very  stringent  test  for  warping.  Many 
anatomical  details  are  evident,  and  even  a  small  mismatch  can  clearly  be  seen.  With  a 
sufficient  number  of  control  points,  the  TPS  transformation  excellently  approximated  the 
deformations  of  the  pelvis  and  internal  structures  of  our  MR  images.  Even  when  we 
warped  the  volume  in  the  diagnostic  position  to  one  in  the  treatment  position,  most 
organs  were  closely  aligned,  despite  very  significant  movements. 

With  warping  registration,  we  have  to  be  concerned  about  potential  warping  errors 
affecting  the  application  of  interest.  For  the  prostate,  we  used  only  one  CP  at  the  prostate 
center  because  we  desired  to  maintain  the  spatial  integrity  of  the  organ  and  to  preserve  the 
tissue  volume.  We  placed  many  CP’s  around  the  pelvic  surface  to  produce  reasonable 
warping. 

The  semi-automatic  method  is  fast  and  reliable.  A  user  only  needs  to  place  5  feature 
points  in  a  procedure  that  is  straightforward  with  our  graphical  user  interface.  All  other 
tasks  are  automatic.  As  compared  to  the  manual  warping  method,  the  almost  fully 
automated  method  saved  time  and  labor,  and  gave  registrations  that  were  as  good  as  or 
better  than  the  manual  method. 

The  flexibility  of  the  semi-automatic  method  makes  the  current  software  suitable  for 
warping  registration  in  many  applications  in  addition  to  the  clinical  procedures  described 
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in  the  Introduction.  We  believe  that  the  registration  method  can  be  applied  to  other 
modalities  and  inter-subject  images.  In  addition,  the  general  approach  can  probably  be 
modified  for  other  organ  systems. 


B.  Algorithmic  Robustness  and  Efficiency 

For  rigid  body  registration,  two  principal  design  features  greatly  improved  the 
robustness  of  the  global  registration  [24].  First,  using  both  CC  and  MI  at  different 
resolutions  was  an  important  feature  that  increased  robustness.  CC  gave  fewer  local 
minimums  at  low  resolutions  and  MI  was  more  accurate  at  high  resolution  [44][45].  Our 
registration  algorithm  combined  these  advantages  from  the  two  similarity  measures. 
Second,  the  restarting  mechanism  was  important.  Without  restarting,  we  found  that 
registrations  sometimes  failed  in  cases  of  volumes  with  large  mismatches  and  significant 
deformation.  Even  these  cases  resulted  in  a  proper  solution  when  restarting  was 
employed. 

The  warping  registration  algorithm  is  designed  to  be  very  computationally  efficient 
for  TPS  warping  with  hundreds  of  CP's.  First,  the  optimization  of  small  VOI's  is  very  fast. 
Second,  we  optimized  each  CP  separately  because  the  optimization  of  six  parameters 
(three  translations  and  three  rotations)  is  much  simple  and  fast.  Conversely,  the 
optimization  of  many  CP's  at  the  same  time  [51]  leads  to  a  much  more  complicated  error 
surface  and  local  maximums.  If  one  were  to  use  600  CP's  and  optimizes  the  1800  free 
parameters  simultaneously,  the  optimization  process  would  become  extraordinarily 
complex.  Third,  we  applied  the  TPS  transformation  once  to  the  final  optimal  CP's;  this 
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saved  considerable  time  and  gave  satisfactory  warping.  If  TPS  were  applied  during  each 
iteration  of  the  optimization  process  [51],  the  registration  time  would  be  quite 
unacceptable.  If  we  were  to  use  optimized  C  code  instead  of  IDL,  the  rigid  body  and 
warping  registration  should  be  more  time  efficient. 


C.  Evaluation  of  Warping  Registration 

Since  there  is  no  standard  method,  we  used  a  variety  of  methods  to  evaluate  warping 
registration  quality.  First,  for  routine  evaluation,  a  color  overlay  is  simple,  fast,  and 
intuitive.  To  better  visualize  the  two  data  sets,  we  interactively  adjust  the  transparency 
scale  of  each  image.  Second,  for  illustration  of  subtle  difference  along  an  edge,  we  use  a 
sector  display  because  it  best  shows  small  shifts  [44].  Third,  for  visual  evaluation  of  a 
specific  organ  such  as  the  prostate,  we  like  to  superimpose  manually  marked  contours 
from  one  image  onto  another  as  shown  in  Figure  7.  This  clearly  shows  any  displacement 
or  deformation.  Fourth,  a  more  quantitative  approach  is  obtained  by  calculating  die 
displacement  of  a  3D  centroid  of  the  segmented  prostate.  Finally,  when  images  have 
comparable  gray  levels,  a  difference  image,  and  statistics  on  the  difference  image, 
provides  yet  another  approach.  A  downside  with  MR  images  is  that  the  inhomogeneity  of 
the  signal  response  and  interpolation  can  introduce  artifacts  in  the  difference  images. 
Since  MR  image  intensity  can  vary  with  different  MR  sequence  parameters  and  the  signal 
response  of  MR  coil,  the  gray  value  statistic  may  have  some  limitations  when  image 
acquisitions  are  not  carefully  repeated.  Similarly,  there  are  other  gray-level  measures  such 
as  the  correlation  coefficient.  Although  the  absolute  value  of  CC,  and  other  gray-scale 
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measures,  might  not  predict  registration  quality,  it  is  probably  a  very  good  way  to 
compare  registrations  on  a  single  volume  pair,  as  done  in  Figure  9. 

We  conclude  that  the  semi-automatic  warping  registration  works  better  than  rigid 
body  registration  when  patient  position  is  greatly  changed  between  acquisitions.  It  also 
compares  favorably  to  warping  registration  with  manual  selection  of  control  points.  We 
believe  it  will  be  a  useful  tool  for  many  applications. 


ACKNOWLEDGEMENTS 

The  algorithm  developed  in  this  research  was  supported  by  NIH  grant  R01-CA84433- 
01  to  David  L.  Wilson  and  DOD  grant  DAMD 17-02- 1-0230  to  Baowei  Fei.  Imaging 
techniques  were  developed  under  the  support  of  NIH  grant  R33-CA88 144-01  to  Jeffrey  L. 
Duerk. 


53 


24 


REFERENCE  LIST 


[1]  A.Carrillo,  J.L.Duerk,  J.S.Lewin,  and  D.L. Wilson,  "Semiautomatic  3-D  image 
registration  as  applied  to  interventional  MRI  liver  cancer  treatment,"  IEEE 
Transactions  on  Medical  Imaging,  vol.  19,  pp.  175-185,  2000. 

[2]  J.S.Lewin,  C.F.Connell,  J.L.Duerk,  Y.C.Chung,  M.E.Clampitt,  J.Spisak, 
G.S.Gazelle,  and  J.R.Haaga,  "Interactive  MRI-guided  radiofrequency  interstitial 
thermal  ablation  of  abdominal  tumors:  Clinical  trial  for  evaluation  of  safety  and 
feasibility,"  Journal  of  Magnetic  Resonance  Imaging,  vol.  8,  pp.  40-47, 1998. 

[3]  E.M.Merkle,  J.R.Shonk,  J.L.Duerk,  G.H.Jacobs,  and  J.S.Lewin,  "MR-guided  RF 
thermal  ablation  of  the  kidney  in  a  porcine  model,"  American  Journal  of 
Roentgenology,  vol.  173,  pp.  645-651, 1999. 

[4]  K.Kagawa,  W.R.Lee,  T.E.Schultheiss,  M.A.Hunt,  A.H.Shaer,  and  G.E.Hanks, 
"Initial  clinical  assessment  of  CT-MRI  image  fusion  software  in  localization  of  the 
prostate  for  3D  conformal  radiation  therapy,"  International  Journal  of  Radiation 
Oncology  Biology  Physics,  vol.  38,  pp.  319-325, 1997. 

[5]  M.Milosevic,  S.Voruganti,  R.Blend,  H.Alasti,  P.Warde,  M.McLean,  P.Catton, 
C.Catton,  and  M.Gospodarowicz,  "Magnetic  resonance  imaging  (MRI)  for 
localization  of  the  prostatic  apex:  comparison  to  computed  tomography  (CT)  and 
urethrography,"  Radiotherapy  and  Oncology,  vol.  47,  pp.  277-284, 1998. 

[6]  M.Roach,  P.FaillaceAkazawa,  C.Malfatti,  J.Holland,  and  H.Hricak,  "Prostate 
volumes  defined  by  magnetic  resonance  imaging  and  computerized  tomographic 
scans  for  three-dimensional  conformal  radiotherapy,"  International  Journal  of 
Radiation  Oncology  Biology  Physics,  vol.  35,  pp.  101 1-1018, 1996. 

[7]  C.Rasch,  LBarillot,  P.Remeijer,  A.Touw,  M.van  Herk,  and  J.VXebesque, 
"Definition  of  the  prostate  in  CT  and  MRI:  A  multi-observer  study,"  International 
Journal  of  Radiation  Oncology  Biology  Physics,  vol.  43,  pp.  57-66,  1999. 

[8]  R.A.H.Boni,  J.A.Boner,  J.F.Debatin,  F.Trinkler,  H.Knonagel,  A.Vonhochstetter, 
U.Helfenstein,  and  G.P.Krestin,  "Optimization  of  prostate  carcinoma  staging  - 
comparison  of  imaging  and  clinical  methods,"  Clinical  Radiology,  vol.  50,  pp.  593- 
600,  1995. 

[9]  D.B.Sodee,  G.T.MacLennan,  M.I.Resnick,  PJF.Faulhaber,  Z.Lee,  A.D.Nelson, 
J.P.Molter,  and  G.Bakale,  "Comparison  of  CT-  or  MRI-fused  PET-FDG  and 
SPECT-ProstaScint  (R)  imaging  of  prostate  cancer  with  the  gold  standard  of 
histology,"  Journal  of  Nuclear  Medicine,  vol.  42,  pp.  1222,2001. 

[10]  J.Scheidler,  H.Hricak,  D.B.Vigneron,  K.K.Yu,  D.L.Sokolov,  L.R.Huang, 
C.J.Zaloudek,  S.J.Nelson,  P.R.Carroll,  and  J.Kurhanewicz,  "Prostate  cancer: 
Localization  with  three-dimensional  proton  MR  spectroscopic  imaging  - 
Clinicopathologic  study,"  Radiology,  vol.  213,  pp.  473-480, 1999. 


54 


25 


[11]  H.Gray.  Anatomy,  Descriptive  and  Surgical  (The  classic  collector's  edition).  New 
York:  Gramercy  Books,  1977. 

[12]  M.vanHerk,  A.Bruce,  A.P.G.Kroes,  T.Shouman,  A.Touw,  and  J.V.Lebesque, 
"Quantification  of  organ  motion  during  conformal  radiotherapy  of  the  prostate  by 
three  dimensional  image  registration,"  International  Journal  of  Radiation  Oncology 
Biology  Physics,  vol.  33,  pp.  1311-1320,  1995. 

[13]  R.K.Tenhaken,  J.D.Forman,  D.K.Heimburger,  A.Gerhardsson,  D.L.Mcshan, 
C.Pereztamayo,  S.L.Schoeppel,  and  A.S.Lichter,  "Treatment  planning  issues 
related  to  prostate  movement  in  response  to  differential  filling  of  the  rectum  and 
bladder,"  International  Journal  of  Radiation  Oncology  Biology  Physics,  vol.  20, 
pp. 1317-1324, 1991. 

[14]  J.CXiehn,  A.Loboguerrero,  X.Perault,  and  L.Demange,  "Superimposition  of 
computed-tomography  and  single  photon  emission  tomography  immunoscinti- 
graphic  images  in  the  pelvis  -  validation  in  patients  with  colorectal  or  ovarian 
carcinoma  recurrence,"  European  Journal  of  Nuclear  Medicine,  vol.  19,  pp.  186- 
194, 1992. 

[15]  J.M.Balter,  H.M.Sandler,  K.Lam,  R.L.Bree,  A.S.Lichter,  and  R.K.Ten  Haken, 
"Measurement  of  prostate  movement  over  the  course  of  routine  radiotherapy  using 
implanted  markers,"  International  Journal  of  Radiation  Oncology  Biology  Physics, 
vol.  31,  pp.  113-118, 1995. 

[16]  V.Narayana,  P.L.Roberson,  R.J. Winfield,  and  P.W.McLaughlin,  "Impact  of 
ultrasound  and  computed  tomography  prostate  volume  registration  on  evaluation  of 
permanent  prostate  implants,"  International  Journal  of  Radiation  Oncology 
Biology  Physics,  vol.  39,  pp.  341-346, 1997. 

[17]  M.van  Herk,  J.C.de  Munck,  J.V.Lebesque,  S.Muller,  C.Rasch,  and  A.Touw, 
"Automatic  registration  of  pelvic  computed  tomography  data  and  magnetic 
resonance  scans  including  a  dull  circle  method  for  quantitative  accuracy 
evaluation,"  Medical  Physics,  vol.  25,  pp.  2054-2067, 1998. 

[18]  J.A.Antolak,  I.I.Rosen,  C.H.Childress,  G.K.Zagars,  and  A.Pollack,  "Prostate  target 
volume  variations  during  a  course  of  radiotherapy,"  International  Journal  of 
Radiation  Oncology  Biology  Physics,  vol.  42,  pp.  661-672, 1998. 

[19]  P.Remeijer,  E.Geerlof,  L.Ploeger,  K.Gilhuijs,  M.van  Herk,  and  J.V.Lebesque,  "3-D 
portal  image  analysis  in  clinical  practice:  An  evaluation  of  2-D  and  3-D  analysis 
techniques  as  applied  to  30  prostate  cancer  patients,"  International  Journal  of 
Radiation  Oncology  Biology  Physics,  vol.  46,  pp.  1281-1290, 2000. 

[20]  J.C.Roeske,  J.D.Forman,  C.F.Mesina,  T.He,  C.A.Pelizzari,  E.Fontenla, 
S.Vijayakumar,  and  G.T.Y.Chen,  "Evaluation  of  changes  in  the  size  and  location  of 
the  prostate,  seminal  vesicles,  bladder,  and  rectum  during  a  course  of  external  beam 
radiation  therapy,"  International  Journal  of  Radiation  Oncology  Biology  Physics 
vol.  33,  pp.  1321-1329, 1995. 


55 


26 


[21]  A.M.Scott,  H.A.Macapinlac,  C.R.Divgi,  J.LZhang,  H.Kalaigian,  K.Pentlow, 
S. Hilton,  Graham.M.C,  G.Sgouros,  C.Pelizzari,  G.Chen,  J.Schlom,  S.J.Goldsmith, 
and  S.M.Larson,  "Clinical  validation  of  SPECT  and  CT/MRI  image  registration  in 
radiolabeled  monoclonal-antibody  studies  of  colorectal  carcinoma,"  Journal  of 
Nuclear  Medicine,  vol.  35,  pp.  1976-1984, 1994. 

[22]  RJ.Hamilton,  MJ.Blend,  C.A.Pelizzari,  B.D.Milliken,  and  S.Vijayakumar,  "Using 
vascular  structure  for  CT-SPECT  registration  in  the  pelvis,"  Journal  of  Nuclear 
Medicine,  vol.  40,  pp.  347-351, 1999. 

[23]  M.V.Herk,  J.C.de  Munck,  J.V.Lebesque,  S.Muller,  C.Rasch,  and  A.Touw, 
"Automatic  registration  of  pelvic  computed  tomography  data  and  magnetic 
resonance  scans  including  a  full  circle  method  for  quantitative  accuracy  evaluation," 
Medical  Physics,  vol.  25,  pp.  2054-2067,  1998. 

[24]  B.W.Fei,  A. Wheaton,  Z.Lee,  J.L.Duerk,  and  D.L.Wilson,  "Automatic  MR  volume 
registration  and  its  evaluation  for  the  pelvis  and  prostate,"  Physics  in  Medicine  and 
Biology,  vol.  47,  pp.  823-838,  2002. 

[25]  C.R.Meyer,  J.L.Boes,  B.Kim,  P.H.Bland,  G.L.LeCarpentier,  J.B.Fowlkes, 
N.A.Roubidoux,  and  P.L.Carson,  "Semiautomatic  registration  of  volumetric 
ultrasound  scans,"  Ultrasound  in  Medicine  and  Biology,  vol.  25,  pp.  339-347 
1999. 

[26]  J.F.Krucker,  C.R.Meyer,  G.L.LeCarpentier,  IB.Fowlkes,  and  P.L.Carson,  "3D 
spatial  compounding  of  ultrasound  images  using  image-based  nonrigid  registration," 
Ultrasound  in  Medicine  and  Biology,  vol.  26,  pp.  1475-1488, 2000. 

[27]  M.H.Davis,  A.Khotanzad,  DJP.Flamig,  and  S.E.Harms,  "A  physics-based 
coordinate  transformation  for  3-D  image  matching,"  IEEE  Transactions  on 
Medical  Imaging,  vol.  16,  pp.  317-328, 1997. 

[28]  G.E.Christensen,  R.D.Rabbitt,  and  M.LMiller,  "Deformable  templates  using  large 
deformation  kinematics,"  IEEE  Transactions  on  Image  Processing,  vol.  5,  pp. 
1435-1447,1996. 

[29]  K.Rohr,  H.S.Stiehl,  R.Sprengel,  T.MXuzug,  J.Weese,  and  M.H.Kuhn,  "Landmark- 
based  elastic  registration  using  approximating  thin-  plate  splines,"  IEEE 
Transactions  on  Medical  Imaging,  vol.  20,  pp.  526-534, 2001. 

[30]  D.Rueckert,  L.I.Sonoda,  C.Hayes,  D.L.G.Hill,  M.O.Leach,  and  DJ.Hawkes, 
"Nonrigid  registration  using  free-form  deformations:  Application  to  breast  MR 
images,"  IEEE  Transactions  on  Medical  Imaging,  vol.  18,  pp.  712-721, 1999. 

[31]  S.K.Kyriacou  and  C.Davatzikos,  "A  biomechanical  model  of  soft  tissue 
deformation,  with  applications  to  non-rigid  registration  of  brain  images  with  tumor 
pathology,"  Medical  Image  Computing  and  Computer-Assisted  Intervention  - 
Miccai'98,  vol.  1496,  pp.  531-538, 1998. 


56 


27 


[32]  S.Kimiaei,  M.Noz,  EJonsson,  J.Crafoord,  and  G.Q.Maguire,  "Evaluation  of 
polynomial  image  deformation  using  anatomical  landmarks  for  matching  of  3D- 
abdominal  MR-images  and  for  atlas  construction,"  IEEE  Transactions  on  Nuclear 
Science,  vol.  46,  pp.  1 1 10-1 1 13, 1999. 

[33]  M.Otte,  "Elastic  registration  of  fMRI  data  using  Bezier-spline  transformations," 
IEEE  Transactions  on  Medical  Imaging,  vol.  20,  pp.  193-206, 2001. 

[34]  G.E.Christensen,  P.Yin,  M.W.Vannier,  K.S.C.Chao,  J.F.Dempsey,  and 
J.F. Williamson,  "Large-deformation  image  registration  using  fluid  landmarks," 
Proceedings  of  4th  IEEE  Southwest  Symposium  on  Image  Analysis  and 
Interpretation,  pp.  269-273, 2000. 

[35]  C.R.Maurer,  D.L.G.Hill,  A.J.Martin,  H.Y.Liu,  M.McCue,  D.Rueckert,  D.Lloret, 
W.A.Hall,  R.E.Maxwell,  D.J.Hawkes,  and  C.L.Truwit,  "Investigation  of 
intraoperative  brain  deformation  using  a  1.5-t  interventional  MR  system: 
Preliminary  results,"  IEEE  Transactions  on  Medical  Imaging,  vol.  17,  pp.  817-825 
1998. 

[36]  E.R.E.Denton,  L.I.Sonoda,  D.Rueckert,  S.C.Rankin,  C.Hayes,  M.O.Leach, 
D.L.G.Hill,  and  D.J.Hawkes,  "Comparison  and  evaluation  of  rigid,  affine,  and 
nonrigid  registration  of  breast  MR  images,"  Journal  of  Computer  Assisted 
Tomography,  vol.  23,  pp.  800-805, 1999. 

[37]  D.L. Wilson,  A.Carrillo,  L.Zheng,  A.Genc,  J.L.Duerk,  and  J.S.Lewin,  "Evaluation 
of  3D  image  registration  as  applied  to  MR-guided  thermal  treatment  of  liver 
cancer,"  Journal  of  Magnetic  Resonance  Imaging,  vol.  8,  pp.  77-84, 1998. 

[38]  A.Bharatha,  M.Hirose,  N.Hata,  S.K.Warfield,  M.Ferrant,  K.H.Zou,  E.Suarez- 
Santana,  J.Ruiz-Alzola,  A.D'Amico,  R.A.Cormack,  R.Kikinis,  F.A.Jolesz,  and 
C.M.C.Tempany,  "Evaluation  of  three-dimensional  finite  element-based 
deformable  registration  of  pre-  and  intraoperative  prostate  imaging,"  Medical 
Physics,  vol.  28,  pp.  2551-2560, 2001. 

[39]  B.W.Fei,  C.Kemper,  and  DX.Wilson,  "A  comparative  study  of  warping  and  rigid 
body  registration  for  the  prostate  and  pelvic  MR  volumes,"  Computerized  Medical 
Imaging  and  Graphics,  (In  Press),  2002. 

[40]  B.W.Fei,  C.Kemper,  and  D.L. Wilson,  "Three-dimensional  warping  registration  of 
the  pelvis  and  prostate,"  Proceedings  of  SPIE  Medical  Imaging  on  Image 
Processing,  vol.  4684,  pp.  528-537, 2002. 

[41]  Z.Lee,  C.Kemper,  RF.Muzic,  M.S.Berridge,  and  D.L. Wilson,  Quantitative 

pulmonary  imaging  based  on  PET-CT  co-registration  with  warping  Journal  of 
Nuclear  Medicine,  vol.  42,  Suppl,  pp.  10-11,2001. 

[42]  A.Collignon,  F.Maes,  D.Delaere,  D.Vandermeulen,  P.Suetens,  and  G.Marchal, 
"Automated  multimodality  image  registration  using  information 
theory  "Information  Processing  in  Medical  Imaging:  Proc.  14th  International 
Conference  (IPMI'95),  Computational  Imaging  and  Vision,  pp.  287-298, 1995. 


57 


28 


[43]  W.H.Press,  S.A.Teukolsky,  W.T.Vellerling,  and  B.P.Flannery.  Numerical  Recipes 
in  C:  The  Art  of  Scientific  Computing,  Second  Edition.  New  York:  The  Press 
Syndicate  of  the  Cambridge  University,  1992. 

[44]  B.W.Fei,  JX.Duerk,  D.T.Boll,  J.S.Lewin,  and  D.L.Wilson,  "Slice  to  volume 
registration  and  its  potential  application  tointerventional  MRI  guided 
radiofrequency  thermal  ablation  of  prostate  cancer,"  IEEE  Transactions  on  Medical 
Imaging,  (In  Press),  2002. 

[45]  B.W.Fei,  A. Wheaton,  Z.Lee,  K.Nagano,  J.L.Duerk,  and  D.L. Wilson,  "Robust 
registration  algorithm  for  interventional  MRI  guidance  for  thermal  ablation  of 
prostate  cancer,"  Proceedings  of  SPIE  Medical  Imaging  on  Visualization,  Display, 
and  Image-Guided  Procedures,  vol.  4319,  pp.  53-60,  2001. 

[46]  J.P.W.Pluim,  J.B.A.Maintz,  and  M.A.Viergever,  "Image  registration  by 
maximization  of  combined  mutual  information  and  gradient  information,"  IEEE 
Transactions  on  Medical  Imaging,  vol.  19,  pp.  809-814, 2000. 

[47]  J.Nelder  and  R.A.Mead,  "A  simplex  method  for  function  minimization,"  Computer 
Journal,  vol.  7,  pp.  308-313,  1965. 

[48]  A.Goshtasby,  "Registration  Of  Images  With  Geometric  Distortions,"  IEEE 
Transactions  on  Geoscience  and  Remote  Sensing,  vol.  26,  pp.  60-64,  1988. 

[49]  F.L.Bookstein,  "Principal  warps  -  thin-plate  splines  and  the  decomposition  of 
deformations,"  IEEE  Transactions  on  Pattern  Analysis  and  Machine  Intelligence, 
vol.  11,  pp.  567-585,  1989. 

[50]  F.L.Bookstein,  "Thin-plate  splines  and  the  atlas  problem  for  biomedical  images," 
Lecture  Notes  in  Computer  Science,  vol.  5 1 1,  pp.  326-342, 1991 . 

[51]  C.R.Meyer,  J.L.Boes,  B.Kim,  P.H.Bland,  K.R.Zasadny,  P.V.Kison,  K.Koral, 
K.A.Frey,  and  R.L.Wahl,  "Demonstration  of  accuracy  and  clinical  versatility  of 
mutual  information  for  automatic  multimodality  image  fusion  using  affine  and  thin- 
plate  spline  warped  geometric  deformations,"  Medical  Image  Analysis,  vol.  1 ,  pp 
195-206, 1996. 


58 


29 


FIGURE  LEGENDS 

Figure  1.  Flow  chart  of  the  warping  registration  algorithm.  The  first  four  steps  are  to 
create  features  points  in  both  volumes  following  global  rigid  body  registration.  The  loop 
from  FOR  to  END  is  to  optimize  the  volumes  of  interest  (VOI)  as  centered  at 
corresponding  feature  points  using  mutual  information.  The  optimal  VOI  is  then  used  to 
determine  nine  control  points  that  represent  the  position  and  orientation  of  the  VOI.  The 
position  information  of  all  control  points  is  used  to  establish  a  three-dimensional  TPS 
transformation  for  the  entire  volume.  See  text  for  details. 

Figure  2.  Locations  of  the  manually  selected  five  feature  points.  FP0  is  at  the  prostate 
center;  FPi  and  FP2,  the  hip  joints;  and  FP3  and  FP4;  the  distal  femurs.  The  dash  lines 
connecting  FPi  and  FP3,  and  FP2  and  FP4  are  along  the  femurs. 

Figure  3.  Automatic  detection  of  the  pelvic  surfaces.  Image  (a)  is  a  transverse  image  slice 
covering  the  prostate.  The  dash  lines  pass  thought  the  prostate  with  an  equal  angle 
increment.  The  dots  are  at  the  pelvic  exterior  surfaces.  Graph  (b)  plots  the  intensity 
signals  along  one  dash  line  in  (a).  A  threshold  T  is  used  to  determine  the  pelvic  surfaces 
along  this  line. 

Figure  4.  Nine  control  points  for  each  feature  point  and  volume  of  interest  (VOI).  The 
big  cubic  volume  is  the  VOI  that  is  used  to  calculate  mutual  information  and  optimize  the 
feature  point  FP.  The  small  cubic  volume  is  with  die  same  center  and  the  same  orientation 
but  half  size  on  a  side.  The  center  and  the  eight  comers  of  the  smaller  cubic  volume  are 
the  control  points  that  will  be  used  to  establish  thin  plate  spline  transformation. 
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Figure  5.  Feature  points  for  images  acquired  in  the  treatment  and  diagnostic  positions. 
Image  (a)  is  from  the  reference  volume  acquired  in  the  treatment  position  with  legs 
raised.  Image  (b)  is  to  be  warped  and  is  from  the  volume  acquired  in  the  diagnostic 
position  with  legs  flat  on  the  table.  Feature  points  are  located  at  the  prostate  center, 
femurs,  and  pelvic  surfaces.  Each  feature  point  is  located  at  one  voxel  but  displayed  much 
larger  for  improved  visualization.  Volumes  are  of  volunteer  S2. 

Figure  6.  Mutual  information  as  a  function  of  angle  increment.  The  y-axis  was  MI  values 
between  registered  volumes.  The  x-axis  was  the  angle  increments  that  were  used  to  detect 
pelvic  surfaces.  Generally,  MI  values  decreased  when  the  angle  increment  increased 
because  less  FP's  participated  warping  registration.  MI  values  saturate  when  the  angle 
increment  is  smaller  than  20  degree.  The  number  of  FP’s  varied  from  15  to  60  for  angle 
increments  of  1 07  to  43 ,  respectively. 

Figure  7.  Comparison  of  warping  and  rigid  body  registration  for  volumes  acquired  in  the 
treatment  and  diagnostic  positions.  Image  (a)  is  from  the  reference  volume  acquired  in  the 
treatment  position,  and  the  prostate  is  manually  segmented.  Images  in  the  left  and  right 
columns  are  from  the  floating  volume  acquired  in  the  diagnostic  position  following  rigid 
body  and  warping  registration,  respectively.  To  show  potential  mismatch,  the  prostate 
contour  from  the  reference  in  (a)  is  copied  to  (b)  and  (c)  and  magnified  as  the  dashed 
contours  in  (d)  and  (e).  The  movement  of  the  prostate  to  the  posterior  is  corrected  with 
warping  (e)  but  not  rigid  body  registration  (d).  Pelvic  boundaries  manually  segmented 
from  the  reference  show  significant  misalignment  with  rigid  body  (f)  that  is  greatly 
improved  with  warping  (g).  Images  are  transverse  slices  from  volunteer  S2. 
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Figure  8.  Prostate  centroid  displacement  following  rigid  body  and  warping  registration. 
Ten  treatment-diagnosis  volume  pairs  from  three  volunteers  were  registered  using  rigid 
body  and  warping  registration,  respectively.  Following  rigid  body  registration,  the  mean 
displacement  of  prostate  centroid  between  registered  volumes  was  4.9  ±  3.4  mm  cross  all 
volume  pairs.  The  latter  number  is  the  standard  deviation.  Following  warping,  the  mean 
displacement  was  only  0.7  ±  0.2  mm.  Warping  significantly  decreased  the  displacement 
indicating  much  better  registration  of  the  prostate. 

Figure  9.  Comparison  of  rigid  body,  semi-automatic,  and  manual  warping  registration  for 
treatment-diagnosis  volume  pairs.  Correlation  coefficient  following  registration  is  plotted. 
Both  semi-automatic  and  manual  warping  registration  increased  the  correlation 
coefficient  as  compared  to  rigid  body  registration  in  each  case  and  tins  effect  was 
significant  as  determined  from  all  data  (p  <  0.5%).  The  semi-automatic  method  is  better 
than  the  manual  warping  registration  with  180  control  points  in  7  out  of  10  trials.  There 
are  a  total  10  volume  pairs  from  three  volunteers  as  described  previously. 
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Create  reference  and  floating  volumes  following  global  rigid  body  registration 
Place  five  feature  points  (FP)  at  prostate  center,  hip  joints,  and  femurs  in  both  volumes 
Create  feature  points  along  femur  using  line  segment  approximation 
Create  feature  points  on  skin  surface  using  the  prostate  center  and  radial  lines 
Initialize  the  size  of  volume  of  interest  (VOI)  centered  at  FP’s 
FOR  FP  FROM  0  to  N-l  DO  BEGIN 
Optimize  mutual  information  (MI)  between  the  reference  and  floating  VOI’s 

1.  Transform  the  floating  VOI  using  three  translations  and  three  rotations 

2.  Interpolate  to  get  a  reformatted  VOI 

3.  Calculate  MI  between  the  reference  and  reformatted  VOI’s 

4.  Vary  the  three  translation  and  three  rotation  parameters 

5.  Repeat  steps  1-4  until  meeting  function  tolerance  or  maximum  iteration  number 
Record  the  optimized  FP  coordinates  in  the  floating  volume 

END 

Determine  control  points  (CP)  using  the  feature  points  and  VOI’s 

Calculate  thin  plate  spline  transformation  using  the  reference  and  optimized  CP’s 

Interpolate  the  floating  volume  and  get  a  warped  volume 
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ABSTRACT 

In  this  study,  we  registered  live-time  interventional  magnetic  resonance  imaging 
(iMRI)  slices  with  a  previously  obtained,  high  resolution,  MRI  volume  that  in  turn  can  be 
registered  with  a  variety  of  functional  images,  e.g.  PET,  SPECT,  for  tumor  targeting.  We 
created  and  evaluated  a  slice-to-volume  registration  algorithm  with  special  features  for  its 
potential  use  in  iMRI-guided,  radiofrequency  (RF)  thermal  ablation  of  prostate  cancer. 
The  algorithm  features  included  a  multi-resolution  approach,  two  similarity  measures, 
and  automatic  restarting  to  avoid  local  minima.  Imaging  experiments  were  performed  on 
volunteers  using  a  conventional  1.5  T  MR  scanner  and  a  clinical  0.2  T  C-arm  iMRI 
system  under  realistic  conditions.  Both  high-resolution  MR  volumes  and  actual  iMRI 
image  slices  were  acquired  from  the  same  volunteers.  Actual  and  simulated  iMRI  images 
were  used  to  test  the  dependence  of  slice-to-volume  registration  on  image  noise,  receive 
coil  inhomogeneity,  and  RF  needle  artifacts.  To  quantitatively  assess  registration,  we 
calculated  the  mean  voxel  displacement  over  a  volume  of  interest  between  slice-to- 
volume  registration  and  volume-to-volume  registration,  which  was  previously  shown  to 
be  quite  accurate.  More  than  800  registration  experiments  were  performed.  For  transverse 
image  slices  covering  the  prostate,  the  slice-to-volume  registration  algorithm  was  100% 
successful  with  an  error  of  <  2  mm,  and  the  average  and  standard  deviation  was  only 
0.4  mm  ±  0.2  mm.  Visualizations  such  as  combined  sector  display  and  contour  overlay 
showed  excellent  registration  of  the  prostate  and  other  organs  throughout  the  pelvis.  Error 
was  greater  when  an  image  slice  was  obtained  at  other  orientations  and  positions,  mostly 
because  of  inconsistent  image  content  such  as  that  from  variable  rectal  and  bladder 
filling.  These  preliminary  experiments  indicate  that  MR  slice-to-volume  registration  is 
sufficiently  accurate  to  aid  image-guided  therapy. 


Keywords:  Medical  imaging,  image  registration,  mutual  information,  magnetic  resonance 
imaging  (MRI),  interventional  MRI,  prostate  cancer,  minimally  invasive  treatment, 
thermal  ablation. 
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I.  INTRODUCTION 

We  use  an  interventional  magnetic  resonance  imaging  (iMRI)  system  to  guide 
minimally  invasive  treatments,  including  the  radiofrequency  (RF)  thermal  ablation  of 
abdominal  cancers  [l]-[3].  The  iMRI  system  consists  of  a  0.2  T,  clinical  C-arm  open  MRI 
scanner,  an  in-room  RF-shielded  liquid  crystal  monitor,  an  MR  compatible  mouse,  a  foot 
pedal,  and  a  RF  device.  We  are  currently  investigating  the  extension  of  these  techniques 
to  the  treatment  of  prostate  cancer.  Since  MRI  does  not  reliably  show  prostate  tumors,  we 
intend  to  incorporate  nuclear  medicine  or  MR  spectroscopy  images  with  higher  sensitivity 
for  detecting  and  localizing  prostate  tumors  [4][5],  We  will  first  register  the  low- 
resolution  functional  images  with  a  high  resolution  MRI  volume  [6][7].  Then  by 
registering  the  high-resolution  MR  volume  with  live-time  iMRI  acquisitions,  we  can,  in 
turn,  map  the  functional  data  and  high-resolution  anatomic  information  to  live-time  iMRI 
images  for  improved  tumor  targeting.  As  discussed  later,  since  live-time  iMRI  is  used  for 
device  guidance,  the  accuracy  requirements  for  registering  these  supplemental  images 
might  be  less  strict  than  required  in  some  other  applications. 

We  previously  described  a  rigid  body  volume-to-volume  (W)  registration  method  for 
the  pelvic  and  prostate  MR  images  that  was  accurate  when  images  were  acquired  under 
similar  conditions  [8].  We  used  bony  landmarks  and  three-dimensional  (3D)  centroids  of 
segmented  prostates  to  evaluate  W  registration.  For  volume  pairs  acquired  over  a  short 
time  span  from  a  supine  subject  with  legs  flat  on  the  table,  registration  accuracy  of  both 
the  prostate  centroid  (typically  <  1  mm)  and  bony  landmarks  (average  1 .6  mm)  was  on ' 
the  order  of  a  voxel  (~  1.4  mm).  The  centroid  error  was  slightly  smaller  because  the 
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prostate  was  at  the  volume  center  and  rotation  errors  had  less  effect  on  it.  The  localization 
error  in  finding  3D  points  from  bony  landmarks  is  probably  greater  than  that  of  finding 
centroids  of  relatively  large  prostate  volumes  where  segmentation  errors  average  out.  We 
obtained  somewhat  larger  prostate  registration  errors  of  about  3.0  mm  when  volume  pairs 
were  obtained  under  very  different  conditions  that  would  be  avoided  in  patient  studies, 
e.g.,  legs  flat  and  legs  raised. 

In  this  study,  we  are  investigating  methods  to  register  live-time  iMRI  image  slices 
with  a  previously  obtained,  high  resolution  MRI  volume.  We  call  this  slice-to-volume 
(SV)  registration.  Because  of  our  success  with  W  prostate  registration,  we  can  determine 
SV  accuracy  by  comparing  results  to  W  registration  for  volume  pairs  having  low  W 
registration  error. 

The  application  of  SV  registration  to  iMRI-guided  treatment  of  prostate  cancer  raises 
several  challenges.  First,  a  single  slice,  or  a  few  slices,  provides  much  less  information 
than  an  entire  volume  for  voxel  based  matching.  Second,  iMRI  images  often  have  lower 
signal  to  noise  ratio  (SNR)  than  diagnostic  MR  images  because  of  the  emphasis  on  fast 
imaging  and  because  of  the  typically  lower  field  strength  of  open  iMRI  magnets.  Third, 
the  normal  prostate  is  a  small  organ;  when  healthy,  it  measures  only  ~  3.8  cm  in  its  widest 
dimension  [9].  The  small  prostate  is  located  below  the  much  larger  bladder  that  can 
change  its  shape  and  size  during  imaging.  Fourth,  the  non-homogenous  receive  coil 
response  can  change  from  one  imaging  session  to  the  next.  Finally,  times  for  registration 
and  algorithm  robustness  are  of  particular  concern  for  this  application  to  treatment. 
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Previously  reported  methods  for  SV  registration  were  mainly  applied  to  the  brain  for 
applications  of  functional  MRI  [10],  postmortem  pathology  studies  [11],  and  anatomical 
modeling  [12].  There  are  no  reports  of  SV  registration  for  abdominal  organs  or  iMRI 
guidance.  Voxel  based  methods,  particularly  those  based  upon  mutual  information  (MI), 
are  robust,  require  no  segmentation  that  can  be  prone  to  error,  are  suitable  for  multi¬ 
modality  registration,  and  are  highly  accurate  for  many  applications  [3][8][10][13]- 
[15].  However,  the  MI  method  has  the  problem  of  interpolation  artifacts,  which  can  be 
especially  serious  in  the  case  of  down  sampling  in  a  multi-resolution  approach  [16].  Other 
similarity  measures  such  as  the  correlation  coefficient  (CC)  can  reduce  the  presence  of 
local  minima  [17]. 

In  this  report,  we  first  describe  a  voxel  based  registration  algorithm  with  special 
features  for  this  important  new  application.  Later,  we  describe  the  details  of  imaging 
experiments  on  a  conventional  MR  scanner  and  a  clinical  iMRI  system.  Actual  and 
simulated  iMRI  images  are  used  to  test  the  registration  algorithm.  Results  of  slice-to- 
volume  and  volume-to-volume  registration  are  compared.  In  this  study,  we  have 
performed  over  800  registration  experiments. 


H.  REGISTRATION  ALGORITHM 

A.  Similarity  Measurements 

We  used  two  similarity  measures,  mutual  information  and  correlation  coefficient,  in 
our  registration.  Suppose  one  image  R  is  the  reference,  and  the  other  F  is  floating.  Their 
mutual  information  MI(R,F)  is  given  below  [18]. 
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mR,F)  =  '£pa,Xr,f)U>s-l 

r,/  P*(r)'PF(f) 

The  joint  probability  p^ir,/) and  the  marginal  probabilities  /^(r)of  the  reference 
image  and  pF{f)  of  the  floating  image,  can  be  estimated  from  the  normalized  joint 
intensity  histogram.  The  correlation  coefficient  CC(R,F )  is  given  below  [19]. 

^vD  S(W-^(r))(F(/)-F(/)) 

)  —  i  —  " "  = _  — 

^(R(r)  -  R(r))2^(F(f)  -  F(f))2 

Here  R(r),  F(f)  denote  the  average  intensities  of  the  reference  and  floating  images  and 
the  summation  includes  all  voxels  within  the  overlap  of  both  images. 

We  compared  the  two  similarity  measures  at  different  resolutions  in  order  to 
determine  their  suitability  for  SV  registration.  At  1/4  resolution,  we  resampled  images  so 
as  to  give  1/4  number  of  the  voxels  along  each  linear  dimension.  At  full  resolution,  we 
used  the  full  number  of  voxels.  In  Figures  1  and  2,  we  plot  the  two  similarity  measures  as 
a  function  of  two  translation  parameters.  After  two  typical  high-resolution  MR  volumes 
were  registered  [8],  values  were  plotted  with  the  origin  as  the  optimal  transformation.  We 
calculated  CC  and  MI  values  while  moving  the  simulated  iMRI  image  relative  to  the 
high-resolution  MR  image  along  coronal  (anterior-posterior)  and  sagittal  (left-right)  axes. 
The  simulated  iMRI  image  was  obtained  as  described  later  in  Section  TIT 

Features  of  MI  and  CC  demonstrate  their  suitability  at  high  and  low  resolutions, 
respectively.  At  1/4  resolution,  CC  surfaces  are  much  smoother  than  MI,  which  is  noisy 
and  contains  many  local  maxima  as  shown  in  Figures  1(a)  and  1(c).  In  fact,  there  is  a  false 
global  maximum  at  +25  voxels.  At  full  resolution,  Figures  2(a)  and  2(c)  shows  that  MI 
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has  a  much  sharper  peak  than  CC,  but  once  again  there  is  high  frequency  noise  in  the  MI 
curves,  far  from  the  optimum,  that  gives  rise  to  local  maxima  that  must  be  avoided.  From 
these  figures,  we  infer  that  CC  is  better  at  low  resolution  and  MI  is  better  at  foil 
resolution,  when  one  is  close  to  the  optimum  value.  As  described  next,  our  registration 
algorithm  makes  use  of  these  features. 

B.  Registration  Algorithm  with  Special  Features 

The  algorithm  includes  special  features  to  improve  robustness  for  registration  of  MR 
prostate  images.  Suppose  the  iMRI  image  slice  is  the  reference  slice,  the  matching  slice 
extracted  from  the  high-resolution  MRI  volume  is  the  reformatted  slice,  and  the  final 
reformatted  slice  is  the  registered  slice.  We  use  a  multi-resolution  approach  and  perform 
registration  from  low  to  high  resolution.  We  use  CC  at  the  two  lower  resolutions  because 
it  gives  fewer  local  maxima  and  because  it  can  be  calculated  faster  than  MI.  We  use  MI  at 
foil  resolution  because  of  its  peaked  surface.  To  avoid  local  maxima,  we  include  a 
restarting  feature  where  registration  is  restarted  with  randomly  perturbed  parameters 
obtained  from  a  uniform  distribution  about  the  initial  transformation  values  at  the  current 
resolution  being  used.  The  algorithm  restarts  until  the  absolute  CC  is  above  a  threshold  of 
0.5  as  determined  later  or  the  maximum  number  of  restarts  is  reached.  Absolute  CC  is 
used  rather  than  MI  because  it  has  a  well-defined  range  between  0  and  1  and  because  it 
provides  an  independent  check  of  the  MI  result  at  the  highest  resolution. 

We  record  all  important  results  following  an  optimization  cycle  including  the  CC 
and/or  MI  values  and  the  transformation  parameters.  At  the  end  of  processing  at  a  lower 
resolution,  we  always  select  the  transformation  parameters  having  the  maximum  CC 
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value.  We  then  scale  the  translation  parameters  appropriately  and  assign  the  new 
parameters  to  be  initial  values  at  the  next  higher  resolution.  At  the  highest  resolution,  MI 
instead  of  CC  is  the  similarity  measure,  and  we  select  the  final  transformation  parameters 
to  be  those  having  the  maximum  MI  value. 

C.  Additional  Details 

Additional  algorithm  details  are  now  described.  For  registration,  we  use  rigid  body 
transformation  (three  translations  and  three  rotations)  and  trilinear  interpolation.  For 
optimization,  we  use  the  downhill  simplex  method  of  Nelder  and  Mead  [20]  or  the 
Powell  method  [21].  Optimization  of  similarity  ends  either  when  the  maximum  number 
of  calculations  is  reached  (typically  500)  or  the  fractional  change  in  the  similarity 
function  is  smaller  than  a  tolerance  (typically  0.001).  The  input  MRI  volume  is  a  3D  MR 
acquisition  giving  256  x  256  x  128  nearly  isotropic  voxels  over  a  field  of  view  covering 
die  whole  pelvis.  We  create  isotropic  voxels  of  about  1.4  mm  on  a  side  using  3D  linear 
interpolation.  We  use  IDL  (Interactive  Data  Language,  Research  System  Inc.,  USA)  as  the 
programming  language. 

Typical  parameter  values  are  now  described.  We  use  an  initial  guess  assuming  an 
identity  transformation,  i.e.,  all  initial  translation  and  rotation  parameters  are  zero, 
because  the  patient  is  normally  oriented  approximately  the  same  way  from  one  scan  to  the 
next.  We  set  the  maximum  numbers  of  restarts  at  10,  5,  and  3,  from  low  to  high 
resolution,  respectively. 
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IE.  EXPERIMENTAL  METHODS 

A.  Image  Acquisition 

High-resolution  MRI  volumes  were  acquired  using  a  1.5  T  Siemens  MRI  system 
(Magnetom  Symphony,  Siemens  Medical  Systems,  Erlangen,  Germany).  An  8-element 
phased  array  body  coil  was  used  to  ensure  coverage  of  the  prostate  with  a  uniform 
sensitivity.  Typically  two  anterior  and  two  posterior  elements  were  enabled  for  signal 
acquisition.  We  used  two  different  MR  sequences.  First,  we  used  a  3D  RF  spoiled 
gradient  echo  steady  state  pulse  sequence  (FLASH)  with  TR/TE/flip  parameters  of 
12/5.0/60  which  give  256  x  256  x  128  voxels  over  a  330  x  330  x  256-mm  field  of  view 
(FOV)  to  yield  1.3  x  1.3x2.0-mm  voxels  oriented  to  give  the  highest  resolution  for 
transverse  slices.  The  acquisition  time  is  5  min  and  38  sec.  This  sequence  is  good  for 
pelvic  imaging  but  is  not  ideal  for  the  prostate.  It  was  used  to  acquire  volumes  for 
volunteer  SI.  Second,  we  used  a  3D  rapid  gradient  echo  sequence  (PSIF)  designed  to 
acquire  the  spin-echo  component  of  the  steady  state  response,  rather  than  the  free 
induction  decay.  The  spin  echo  component  forms  immediately  prior  to  the  RF  pulse;  it  is 
shifted  toward  the  prior  RF  pulse  through  appropriate  gradient  waveform  design.  The 
sequence  with  9.4/5.0/60  (TR/TE/flip)  yields  160x256x128  voxels  over  a 
219  x  350  x  192-mm  rectangular  FOV  and  1.4  x  1.4  x  1.5-mm  voxels  oriented  to  give  the 
highest  resolution  for  transverse  slices.  There  is  over  sampling  at  31%  in  the  slice 
direction  to  reduce  aliasing  artifacts.  The  acquisition  time  is  4  min  and  15  sec.  This 
sequence  gave  excellent  image  contrast  for  the  prostate  and  its  surroundings.  It  was  used 
to  acquire  volumes  for  volunteers  S2-S4. 
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We  also  acquired  iMRI  images  from  the  same  volunteers  using  a  clinical  0.2  T  C-arm 
open  MR  scanner  (Siemens  Open  Symphony,  Erlangen,  Germany)  modified  for 
interventional  MRI  procedures  and  in  this  paper  referred  to  as  the  iMRI  system.  We  used 
a  3D  PSIF  with  25/13/60  (TR/TE/FA)  for  image  volume  acquisitions  and  two- 
dimensional  (2D)  PSIF  with  15.2/7.4/45  (TR/TE/FA)  for  image  slice  acquisitions.  The 
iMRI  volumes  were  256x256x100  with  voxel  size  of  1.3x1. 3x1. 4  mm.  The  iMRI  slices 
were  128x128  with  in-plane  pixel  size  of  2.8x2.8  mm  and  with  effective  slice  thickness 
of  5  mm.  We  acquired  iMRI  images  from  volunteers  S1-S3. 

B.  Simulation  of  iMRI  Image  Slices 

In  experiments,  we  used  high-resolution  MRI  volumes  to  simulate  iMRI  image  slices, 
which  are  thicker,  noisier,  and  degraded  by  receive  coil  inhomogeneity.  Clinically,  we 
typically  use  an  iMRI  slice  thickness  of  4.0  -  6.0  mm.  We  used  trilinear  interpolation  to 
create  isotropic  high-resolution  MRI  volumes  with  voxel  size  of  1.4x1. 4x1. 4  mm.  From 
the  isotropic  high  resolution  MRI  volume,  we  averaged  three  1.4  mm  adjacent  thin  slices 
to  create  a  4.2  mm  thick  slice.  MR  noise  in  a  magnitude  is  described  by  the  Rician 
distribution  [22].  At  SNR  values  of  greater  than  approximately  five,  the  noise  can  be 
approximated  as  being  Gaussian  white  noise  [23].  We  measured  typical  signal  and  noise 
values  on  our  iMRI  system  using  a  homogenous  phantom,  and  volunteer  images  in  the 
region  of  the  prostate  with  methods  described  elsewhere  [24]  [25].  In  all  cases,  image 
SNR  was  greater  than  10  in  all  tissues  including  the  prostate.  With  this  justification,  we 
added  Gaussian  noise  to  the  simulated  iMRI  image  slices  either  to  match  the  measured 
SNR  or  to  give  much  greater  noise  to  further  stress  registration.  We  report  noise 
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experiments  using  the  SNR  of  the  simulated  image  slices.  Figure  3  shows  high-resolution 
MR  images  as  well  as  simulated  and  actual  iMRI  image  slices. 

We  simulated  receive  coil  inhomogeneity  from  a  belt  coil  used  in  our  clinical  iMRI 
acquisitions.  The  coil  is  modeled  as  a  solenoid  with  parameters  shown  in  Figure  4.  Coil 
parameters  are  a,  the  radius  of  the  coil,  2g,  the  length  of  the  coil,  I,  the  current,  jU0 ,  the 

permeability  of  free  space,  n,  the  turns,  and  the  z-axis,  the  axis  along  the  center  line  of  the 
coil.  The  magnetic  field  in  the  xy  plane  can  be  approximated  as  [26]: 


The  z-component  of  the  field  is  given  by  [27]: 

(Ri);  =^(cosa,+cosa2), 
2  g 


(1) 


(2) 


where  the  definition  of  the  angles  a,  and  a2  are  given  in  Figure  4.  The  magnetic  field  is 
highest  at  the  coil  center  and  falls  off  along  the  axial  direction.  According  to  the  Biot- 
Savart  law  [28],  this  model  also  accounts  for  the  spatial  sensitivity  of  the  coil  to  MR 
signal  sources.  Figure  5  shows  a  coronal  image  with  simulated  inhomogeneity  along  the 
axis  (head-foot)  direction. 


Because  a  needle  will  often  be  present  during  an  iMRI  intervention,  we  tested  the 
effect  of  simulated  needles  on  registration.  We  used  artifact  sizes  from  a  previous  report 
on  the  effects  of  pulse  sequence  design  and  magnetic  field  orientation  on  needle  artifacts 
in  MR-guided  biopsy  and  aspiration  [29].  Figure  6  shows  sagittal  images  with  and 
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without  a  simulated  needle  artifact.  The  simulated  artifacts  in  Figure  6(b)  appeared  as 
straight  noisy  bars  2-mm  in  width. 

C.  Imaging  Experiments 

1)  Imaging  Experiments  for  High-Resolution  MR  Volumes 

When  acquiring  high-resolution  MR  volumes,  volunteers  laid  supine  in  a  manner 
similar  to  the  diagnostic  position  in  routine  MR  scanning.  Between  volume  acquisitions, 
volunteers  got  up  from  the  MR  table,  stretched,  and  walked  around  to  ensure  that  they 
would  assume  a  different  position  when  they  laid  back  on  the  table.  Before  the  last  of 
three  volume  acquisitions,  the  volunteer  voided  to  create  an  empty  bladder.  The  coil  array 
was  centered  on  the  prostate.  All  images  of  the  same  volunteer  were  acquired  with  the 
same  MRI  acquisition  parameters.  In  total,  there  are  12  volumes,  3  for  each  of  volunteers 
S1-S4. 

2)  Imaging  Experiments  on  iMRI  System 

We  acquired  iMRI  images  under  the  conditions  simulating  the  treatment  application. 
The  volunteer  was  supine,  and  his  legs  were  supported  at  30°-60°  relative  to  the  horizon 
and  separated  in  a  “V”  with  an  angle  of  60°-90°  between  two  legs.  This  is  similar  to  the 
lithotomy  position  used  in  prostate  therapies,  and  it  should  provide  access  for  needle 
■  insertion  in  brachytherapy  or  RF  thermal  ablation.  We  call  this  the  treatment  position. 
Before  experiments,  the  volunteer  voided  their  bladder.  For  each  volunteer,  all  images 
were  obtained  within  a  two-hour  session.  Between  image  acquisitions,  volunteers  moved 
to  ensure  a  different  position.  For  each  of  the  volunteers  S1-S3,  we  acquired  three 
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volumes  and  50  iMRI  image  slices  covering  the  prostate.  They  included  30  transverse,  10 
coronal,  and  10  sagittal  image  slices.  We  call  these  images  " actual ”  iMRI  images  to 
differentiate  them  from  previous  experiments  using  " simulated ’’  iMRI  slices. 

D.  Registration  Experiments 

1)  Registration  Experiments  Using  Simulated  iMRI  Images 

We  used  12  pairs  of  high-resolution  MR  volumes  to  perform  registration  experiments. 
For  each  volume  pair,  we  extracted  data  from  one  volume  to  simulate  thick  iMRI  image 
slices;  and  then  we  registered  the  simulated  image  slices  to  the  other  volume.  We  desire 
an  iMRI  slice  image  acquisition  method  that  gives  robust,  accurate  registrations  and  is 
relatively  insensitive  to  acquisition  parameters.  Hence,  we  performed  experiments  to 
determine  the  dependence  on  slice  orientation  (transverse,  sagittal  and  coronal),  on  slice 
position  relative  to  the  prostate  (above,  centered,  and  below),  on  image  noise  from  fast 
imaging  techniques,  and  on  the  inhomogeneous  sensitivity  response  from  a  belt  coil. 

2)  Registration  Experiments  Using  Actual  iMRI  Image  Slices 

We  also  performed  two  types  of  SV  registration  experiments  using  the  actual  iMRI 
images.  First,  we  registered  actual  iMRI  image  slices  with  high-resolution  (1.5  T  system) 
MR  volumes  and  visually  evaluated  results.  For  each  volunteer  S1-S3,  there  were  three 
high-resolution  MR  volumes  and  50  iMRI  image  slices  giving  150  SV  registration 
experiments,  and  a  total  of 450  experiments.  Second,  we  registered  thick  slices  simulated 
from  the  volume  of  image  data  obtained  on  the  iMRI  scanner  with  the  corresponding 
high-resolution  (1.5  T  scanner)  MR  volume.  In  this  case,  we  compared  results  to  W 
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registration  obtained  by  registering  the  volume  from  the  iMRI  system  with  the  high- 
resolution  volume  (1.5  T  scanner).  We  investigated  the  effect  of  iMRI  slice  thickness  by 
averaging  1-10  contiguous  image  slices  to  create  a  thick  slice  and  registering  it  to  the 
high-resolution  volume.  The  original  actual  iMRI  volumes  have  a  slice  thickness  of 
1.4  mm  and  in-slice  dimensions  of  1.3  x  1.3  mm.  We  used  trilinear  interpolation  to  create 
isotropic  actual  iMRI  volumes  with  voxel  size  of  1.3xl.3xl.3  mm.  Thus,  thick  slices 
simulated  from  actual  iMRI  volumes  are  1.3  mm  to  13  mm. 

E.  Registration  Evaluation 
1)  Visual  Inspection 

We  evaluated  registration  experiments  by  visual  inspection.  We  used  RegViz,  a 
program  created  in  IDL  in  our  laboratory  with  multiple  visualization  and  analysis 
methods.  First,  we  manually  segmented  prostate  boundaries  in  image  slices  and  copied 
them  to  corresponding  slices.  This  enabled  visual  determination  of  the  overlap  of  prostate 
boundaries  over  the  entire  volume.  Second,  color  overlay  displays  were  used  to  evaluate 
overlap  of  structures.  One  image  was  rendered  in  gray  and  the  other  in  the  “hot-iron” 
color  scheme  available  in  IDL.  To  visualize  potential  differences,  it  was  quite  useful  to 
interactively  change  the  contribution  of  each  image  using  the  transparency  scale.  Third, 
we  used  a  sector  display,  which  divided  the  reference  and  registered  images  into 
rectangular  sectors  and  created  an  output  image  by  alternating  sectors  from  the  two  input 
images.  Even  subtle  shifts  of  edges  would  be  clearly  seen. 


15 


2)  Volume-to-Volume  Registration  Standard 

Our  standard  evaluation  method  was  to  compare  SV  and  W  registration.  The  W 
registration  accuracy  was  previously  evaluated  [8].  For  volume  pairs  acquired  over  a  short 
time  span  from  a  supine  subject  with  legs  flat  on  the  table,  prostates  were  well  aligned 
and  prostate  centroid  displacements  were  typically  <  1  mm.  The  registration  accuracy  as 
determined  from  displacements  of  pelvic  bony  landmarks  was  1.6  mm  ±0.2  mm.  This 
error  might  be  overestimated  because  it  includes  the  uncertainty  of  locating  the  bony 
landmarks.  From  our  success  with  W  prostate  registration,  we  decided  that  we  could 
obtain  SV  accuracy  by  comparing  to  W  registration  for  those  volume  pairs  having  low 
W  registration  error. 

To  compare  SV  and  W  registration  results,  we  defined  a  rectangular  volume  of 
interest  (VOI)  just  covering  the  prostate  over  which  to  calculate  registration  error.  To 
voxels  within  the  VOI,  we  applied  die  transformations  obtained  by  the  W  and  by  SV 
registrations.  We  then  calculated  the  3D  displacements  between  the  transformed  voxels. 
The  mean  voxel  distance  over  the  VOI  was  used  as  our  metric  of  SV  registration  error. 
For  evaluation  of  algorithm  robustness,  we  defined  the  SV  registration  as  being  successful 
when  the  mean  3D  displacement  was  less  than  2.0  mm. 

IV.  RESULTS 

A.  Experiments  with  Simulated  iMRI  Images  from  the  1.5  T  System 

As  described  in  Section  HI,  we  obtained  relatively  low  noise,  high-resolution  MR 
images  and  simulated  SV  registration  results.  These  data  sets  allowed  us  to  test  effects  of 
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noise  and  receive  coil  inhomogeneity  in  a  controlled  fashion.  And,  because  we  had 
substantial  previous  experience  showing  the  accuracy  of  W  registration  under 
comparable  conditions,  we  could  easily  determine  SV  error  by  comparing  results  to  W 
registration. 

In  Figure  7,  the  sector  display  shows  a  simulated  image  slice  registered  with  a  high- 
resolution  image  volume.  The  simulated  image  slice  was  obtained  at  a  transverse 
orientation  near  the  center  of  the  prostate.  The  sector  display  shows  close  alignment  at 
this  position.  Other  transverse  images  were  also  well  aligned  indicating  that  the 
registration  was  successful  in  three  dimensions. 

We  determined  SV  registration  results  for  slices  near  the  prostate  in  the  three  standard 
orthogonal  orientations.  Comparing  to  W,  mean  and  standard  deviation  registration 
errors  across  12  volume  pairs  and  60  SV  registration  experiments  were  0.4  mm  ±  0.2  mm, 
0.5  mm  ±0.2  mm,  and  2.6  mm  ±1.6  mm  for  transverse,  coronal  and  sagittal  slices 
covering  the  prostate,  respectively.  Transverse  slices  worked  best  because  they  contain 
many  relatively  rigid  anatomical  structures  (See  Figure  3).  We  further  found  that 
transverse  slices  centered  on  the  prostate  produced  better  results  than  those  above  or 
below  the  prostate.  Image  slices  above  included  the  deformable  bladder  that  could  give  an 
inconsistent  structure  from  one  volume  to  the  next.  Image  slices  below  the  prostate 
mainly  contained  muscle  and  fatty  regions  from  the  hips  that  could  deform,  again  giving 
inconsistent  image  data.  Coronal  slices  worked  next  best.  Sagittal  slices  gave  the  largest 
error  because  they  contained  a  large  portion  of  the  deformable  bladder  and  rectum. 
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Simulation  experiments  showed  SV  registration  to  be  very  insensitive  to  noise.  We 
performed  over  150  registration  experiments  with  noise  added  to  give  SNR’s  ranging 
from  20  to  5.  Using  the  slice  configurations  recommended  above  (transverse  slices  near 
the  prostate  center),  we  obtained  100%  successful  registrations  (an  error  <  2.0  mm)  for 
SNR’s  =  10,  a  value  much  worse  than  the  clinical  SNR  value  of  ~  25  on  our  iMRI 
system. 

Receive  coil  inhomogeneity  also  had  little  effect  on  registration.  Registration  again 
was  100%  successful  for  all  volume  pairs  under  all  receive  coil  configurations,  even 
when  the  coil  for  the  slice  acquisition  was  displaced  up  to  200  mm  towards  the  head  from 
the  prostate  center,  the  position  of  the  coil  for  the  volume  acquisition. 

B.  Experiments  with  Actual  iMRI  Images 

Figure  8  shows  results  for  an  SV  registration  of  actual  iMRI  image  slices  with  a  high- 
resolution  MR  volume.  The  contours  overlap  and  overlay  images  show  that  the  prostate 
matches  very  well.  Other  visual  inspection  techniques  also  demonstrate  excellent 
registration.  Note  that  a  single  iMRI  image  was  used  to  produce  this  registration  result. 

Figure  9  shows  SV  registration  error  as  a  function  of  slice  thickness.  As  described 
previously,  we  first  registered  each  volume  from  the  iMRI  scanner  with  the  corresponding 
high-resolution  MRI  volume  (1 .5  T  scanner)  using  rigid  body  voxel  based  registration  [8] 
and  used  the  result  as  the  gold  standard  for  calculating  the  SV  error.  Each  thick  slice 
image  was  obtained  by  averaging  several  contiguous  slices  from  the  actual  iMRI  volume. 
As  the  slice  thickness  increases  from  1  x  1.3  mm  to  4  x  1.3  mm,  the  registration  error 
decreases,  possibly  because  of  improved  signal  to  noise  ratio  and/or  because  of  the 
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inclusion  of  more  features.  Error  increases  with  thicker  slices,  probably  because  of  the 
inconsistency  of  image  features  between  the  thick  slice  and  more  finely  sampled  volume. 

In  Figure  10,  we  evaluated  SV  registration  for  thick  slices  at  different  orientations. 
The  evaluation  method  was  the  same  as  that  used  in  Figure  9,  and  the  slices  were  5  mm 
thick  and  intersected  the  volume  near  the  prostate  center.  Results  were  consistent  with 
those  from  the  previous  simulation  experiments.  Transverse  slices  worked  best  with  an 
average  VOI  displacement  of  only  1.1  mm  ±0.7  mm  and  a  success  rate  of  100%.  The 
coronal  images  gave  a  reasonable  average  error,  but  the  success  rate  dropped  to  86%.  The 
sagittal  orientation  gave  the  worst  result. 

Needle  artifacts  had  little  effect  on  the  SV  registration.  In  each  of  30  the  experiments, 
we  registered  a  high-resolution  volume  with  an  actual  iMRI  image  slice  containing  or  not 
containing  a  simulated  needle  artifact.  Visual  inspection,  the  correlation  coefficient  and 
mutual  information  values  of  registered  images  showed  little  effect  of  the  needle  artifact. 
The  success  rate  was  100%  in  both  cases. 

C.  Algorithmic  Robustness  and  Implementation 

The  registration  algorithm  was  quite  robust  for  transverse  slices  covering  the  prostate. 
Using  simulated  iMRI  slices  from  high-resolution  MRI  volume  pairs  of  four  volunteers, 
the  algorithm  never  failed  for  any  transverse  slice  covering  the  prostate.  In  addition,  the 
final  registration  result  was  insensitive  to  initial  guesses  within  a  very  large  range,  [-60, 
+60]  mm  for  translations  and  [-20,  +20]  degrees  for  rotations.  With  the  restarting 
algorithm,  we  even  successfully  registered  slices  as  much  as  80  mm  from  the  optimum. 
This  working  range  should  be  quite  sufficient  for  clinical  applications  where  we  can 
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ensure  good  starting  values.  Using  the  pelvic  bones  as  markers  and  device  localization 
methods  [29],  we  should  be  able  to  position  the  prostate  within  about  ±  20  mm  in  the 
imaging  field.  In  addition,  the  patient  normally  lies  supine  in  the  MR  bed  with  very  little 
rotation  (<  ±  5  degrees). 

Using  CC  and  MI  at  different  resolutions  was  an  important  feature  that  increased 
robustness.  MI  registrations  at  low  resolution  sometimes  gave  false  maxima  (Figure  1(a) 
and  1(c)),  and  only  60%  success  was  achieved  when  MI  was  used  at  all  resolutions.  The 
interpolation  artifacts  at  low  resolutions  often  caused  failures  and  required  more 
restarts  [16].  CC  performed  well  and  gave  fewer  local  maxima  at  the  lower  resolutions 
(Figure  1(b)  and  1(d)),  but  MI  was  more  accurate  than  CC  at  the  highest  resolution  due  to 
the  sharper  peak  of  the  MI  surface  (Figure  2(a)  and  2(c))  [8].  Our  registration  algorithm 
thus  combined  advantages  from  the  two  similarity  measures. 

The  multi-resolution  approach  improved  algorithmic  robustness  and  speed.  When  we 

used  only  MI  at  full  resolution,  registration  was  70%  successful  compared  to  the  100%  of 

\ 

the  full  algorithm.  This  failure  of  MI  was  also  reported  by  others  [13][17].  The  multi¬ 
resolution  approach  enabled  the  program  to  quickly  approach  the  final  value  because  of 
the  reduced  number  of  calculations  at  low  resolutions.  For  a  typical  image  pair,  iterations 
at  1/4  resolution  were  approximately  4  and  25  times  faster  than  at  1/2  and  full  resolution, 
respectively. 

Restarting  was  important  for  image  pairs  with  large  translations  and/or  rotations  from 
the  optimum.  In  our  experience  with  over  800  slice-to-volume  registration  experiments, 
restarting  occurred  iri  about  5%  of  them.  For  an  example  pair  with  an  80  mm 
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displacement,  the  number  of  restarts  was  3,  1,  and  0  at  1/4,  1/2,  and  full  resolutions, 
respectively.  Without  restarting,  we  found  that  registrations  sometimes  failed  in  cases  of 
volumes  with  a  large  mismatch  of  54  mm  and  high  noise.  The  algorithm  was  insensitive 
to  the  CC  threshold  for  restarting.  When  we  decreased  the  threshold  from  0.8  to  0.5  with 
an  interval  of  0.05,  we  found  little  change  in  the  number  of  restarts  and  no  change  in  final 
registrations.  We  set  the  threshold  at  0.5  to  avoid  only  the  most  obvious  local  maxima. 

We  now  describe  some  aspects  of  the  implementation.  The  time  for  an  SV  registration 
was  typically  about  15  sec  on  a  Pentium  IV,  1.8  GHz  CPU,  with  lGbytes  of  memory.  The 
algorithm  was  written  in  IDL  and  could  probably  be  made  much  faster  in  a  lower  level 
language  such  as  C.  A  call  to  the  Simplex  optimization  typically  resulted  in  50  to  105 
similarity  evaluations  before  the  tolerance  value  (0.001)  was  reached.  The  simplex 
optimization  method  worked  about  1. 5-2.0  times  faster  than  the  Powell  method  in  our 
implementation.  We  used  the  Simplex  method  for  our  experiments  in  this  study. 


V.  DISCUSSION  AND  CONCLUSION 

Despite  complications  such  as  image  noise,  receive  coil  inhomogeneity,  a  limited 
number  of  voxels,  and  needle  artifacts,  slice-to-volume  voxel-based  registration  can  be 
quite  robust  and  accurate.  For  transverse  slices  covering  the  prostate,  registration  results 
agreed  very  favorably  with  volume-to-volume  results.  Below,  we  further  discuss  die 
algorithm  and  its  practicality. 
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A.  Mutual  Information  at  Low  Resolution 

There  are  probably  several  reasons  why  mutual  information  does  not  work  well  at  low 
resolution.  First,  the  similarity  curve  is  noisy  with  periodic  oscillations  from  the  so-called 
interpolation  artifact  [8][16]  that  is  accentuated  at  reduced  resolutions  [30].  As  a  result, 
there  are  many  local  maxima  in  Figure  1(a)  and  1(c)  that  can  trap  the  optimization;  and  a 
similar  result  was  reported  for  brain  registration  [13].  In  additional  experiments,  we 
decreased  the  number  of  bins  for  both  images  to  256,  128,  64  and  32  and  plotted  mutual 
information  values  as  a  function  of  translation.  With  a  larger  number  of  bins,  we  got  no 
discemable  effect  of  bin  size.  When  the  number  of  bins  was  reduced  to  32,  the  MI  surface 
was  degraded.  Others  showed  that  Gaussian  blurring  of  images  before  registration  did  not 
improve  performance  at  low  resolutions  and  that  there  was  little  difference  between 
standard  and  normalized  mutual  information  [40].  Second,  when  images  are  of  low 
resolution  and  there  is  only  a  small  region  of  overlap,  the  mutual  information  function  can 
even  contain  incorrect  global  maxima  [30]  as  found  in  Figure  1(a).  This  false  result  was 
obtained  at  very  large  displacements  where  the  slice  to  volume  overlap  was  reduced.  This 
occurs  because  MI  is  not  only  a  function  of  how  well  the  images  match  in  the  overlap,  but 
also  by  how  much  information  is  provided  by  the  two  images  in  the  overlap  [31][32][35]. 
As  shown  above,  using  both  mutual  information  and  correlation  coefficient,  at  different 
resolutions,  was  an  important  feature  that  increased  robustness. 

B.  Accuracy  Consideration 

Essentially,  we  found  that  SV  is  of  similar  accuracy  to  W  registration,  with  an 
average  voxel  displacement  difference  of  only  0.4  mm  in  the  prostate  for  the  simulated 
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images  and  about  1  mm  for  actual  iMRI  image  data.  Hence,  the  accuracy  of  the  best  SV 
method  is  essentially  that  previously  reported  for  W  registration  [8]. 

We  recommend  that  image  data  are  obtained  under  comparable  conditions  by  keeping 
a  similar  posture  and  by  taking  clinical  measures  to  reduce  rectal  and  bladder  filling.  We 
see  no  reason  to  suspect  that  SV  registration  will  be  inaccurate  when  such  conditions  are 
met.  When  images  were  acquired  under  much  different  conditions,  such  as  legs  flat  and 
legs  raised,  rigid  body  registration  could  result  in  prostate  centroid  errors  as  much  as 
3.4  mm.  Another  effect  may  be  the  tissue  deformation  from  insertion  of  the  RF  needle. 
From  our  previous  experience  observing  in  vivo  needle  insertion  in  both  animal  models 
and  clinical  trials  with  real-time  MRJ,  the  amount  of  tissue  deformation  that  occurs  with 
insertion  of  a  sharp  bevel  tip  needle  is  minimal  and  transient  in  tissues  with  normal 
interstitial  pressure.  In  certain  lesions,  such  as  cysts  or  necrotic  tumor,  persistent 
deformation  is  possible;  however,  we  can  see  such  deformations  in  the  live-time 
interventional  MRI  images  and  very  probably  mentally  correct  the  registered,  fused 
images.  We  previously  reported  a  warping  registration  method  [3  8]  [39]  that  can  correct 
deformations  at  the  expense  of  additional  complexity,  time,  and  possibly  robustness. 

The  automatic  slice-to-volume  registration  provides  sufficient  accuracy  for  many 
potential  iMRI  applications.  As  compared  to  a  typical  SPECT  and/or  iMRI  slice  thickness 
of  =  3.0  mm,  SV  registration  is  quite  accurate.  MR  spectroscopy  also  is  done  at  limited 
resolution.  If  one  were  to  use  functional  or  high-resolution  MR  images  directly  for 
targeting,  the  requirements  for  registration  accuracy  would  be  great.  However,  fused 
image  data  will  not  be  used  blindly.  Rather,  these  visualizations  will  be  used  as  a  guide. 
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Physicians  will  always  use  the  live-time  iMRI  images  for  needle  guidance.  With  proper 
visualization  tools,  they  should  be  able  to  mentally  account  for  any  small  registration 
errors.  In  addition,  very  often  there  is  image  evidence  of  cancer  in  MR  prostate  images 
that  can  perhaps  be  identified  with  the  aid  of  functional  images.  Such  MR-visible  lesions 
can  then  become  the  markers  for  tumor  targeting. 

C.  Practicality  and  Application 

The  registration  experiments  presented  here  provided  fairly  comprehensive  tests  for 
the  potential  application  in  iMRI-guided  RF  thermal  ablation  of  the  prostate.  Simulation 
provided  an  efficient  way  to  extensively  evaluate  registration  performance.  The  algorithm 
was  extremely  robust  to  noise  levels,  far  beyond  those  encountered  in  clinical  iMRI 
applications.  Similarly,  the  inhomogeneity  seen  with  a  belt  coil  was  not  problematic  for 
transverse  images,  probably  due  to  coil  inhomogeneity  simply  scaling  the  grayscale 
values,  an  operation  that  should  not  affect  MI  or  CC  similarity  measures.  Needle  artifacts 
had  little  effect,  probably  because  they  occupy  relatively  few  voxels.  The  actual  iMRI 
images  acquired  under  more  realistic  conditions  further  tested  practicality.  Images  from 
the  iMRI  system  contained  more  noise  and  had  less  contrast  than  those  from  the  1.5  T 
scanner.  Registration  quality  was  comparable  to  that  of  simulation  experiments. 
Registration  time  can  probably  be  improved  considerably  using  optimized  C  code  rather 
than  IDL.  If  registration  is  done  in  the  background  in  a  seamless  way,  the  time  for 
registration  is  probably  quite  acceptable.  Although  we  normally  used  T2-weighted  image 
pairs,  the  registration  worked  well  for  pairs  of  Tl-weighted  and  T2-weighted  images. 
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We  conclude  that  the  automatic  slice-to- volume  registration  algorithm  is  quite  robust 
for  transverse  image  slices  covering  the  prostate  and  that  the  registration  provides 
sufficient  accuracy  to  aid  image-guided  therapy.  From  previous  reports  of  MR-PET  or 
MR-SPECT  registration  accuracy  [6]  [7],  it  appears  feasible  to  combine  functional  images 
to  aid  iMRI-guided  procedures.  We  are  beginning  to  explore  this  application  in  animal 
experiments. 
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Figure  Legends 

Figure  1.  Similarity  functions  are  plotted  as  a  function  of  translations  at  the  lowest 
resolution  in  the  multi-resolution  registration  process.  Two  high-resolution  MRI  volumes 
were  registered.  From  the  optimal  parameters,  we  computed  the  similarity  of  the 
simulated  iMRI  and  MRI  images  as  a  function  of  translations  along  the  coronal  (anterior- 
posterior)  and  sagittal  (left-right)  axes.  MI  is  plotted  in  (a)  and  (c);  CC  is  plotted  in  (b) 
and  (d).  Graphs  (a)  and  (b)  are  three-dimensional  (3D)  plots  for  translations  along  the 
coronal  and  sagittal  axis.  Graphs  (c)  and  (d)  are  two-dimensional  (2D)  plots  for 
translations  about  the  coronal  axis.  The  small  insets  in  (c)  and  (d)  are  magnified  curves 
showing  noise  having  local  maxima  in  (c).  A  false  global  maximum  for  MI  occurred  at 
+25  voxels.  Images  are  from  volunteer  S2,  and  they  are  down  sampled  by  1/4  along  each 
linear  dimension,  giving  a  distance  between  voxel  centers  of  ~  5.5  mm. 

Figure  2.  Similarity  functions  are  plotted  as  a  function  of  translations  at  full  resolution. 
Many  details  are  given  in  the  legend  of  Figure  1.  Again,  MI  is  plotted  in  (a)  and  (c);  CC  is 
plotted  in  (b)  and  (d).  MI  in  (a)  and  (c)  has  a  much  sharper  peak  than  CC  in  (b)  and  (d). 
The  voxel  is  isotropic  with  1.4  mm  on  a  side.  Image  data  are  the  same  used  in  Figure  1. 

Figure  3.  High-resolution  MR  images,  simulated  and  actual  iMRI  image  slices.  Images 
on  the  left  column,  (a),  (d)  and  (g),  are  the  original  high-resolution  MR  images  from  the 
1.5  T  scanner  in  the  transverse,  coronal,  and  sagittal  planes,  respectively.  Images  in  the 
middle  column  are  the  corresponding,  simulated  thick  iMRI  images  with  noise  added  to 
give  SNR  =  15  and  with  sensitivity  fall  off  from  a  belt  coil.  Images  on  the  right  panel  are 
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actual  iMRI  slices  (0.2  T  scanner)  from  similar  spatial  locations.  The  actual  iMRI  slices 
seem  blurred  because  of  nearly  doubled  pixel  size.  Images  are  from  volunteer  S2. 

Figure  4.  Geometry  of  solenoidal  receive  coil.  Model  parameters  are  defined  in  the 
figure.  The  axial  line  is  along  the  cranial-caudal  direction  of  the  patient. 

Figure  5.  Simulated  signal  changes  due  to  receive  coil  inhomogeneity.  The  original 
image  (a)  is  acquired  using  a  phased  array  coil  on  a  conventional  1.5  T  MRI  system. 
Using  a  belt  coil  model  with  a  diameter  of  350  mm  and  a  width  of  50  mm,  the  simulated 
iMRI  image  is  shown  in  (b).  The  image  intensity  is  highest  at  the  center  and  decreases 
along  the  axial  direction. 

Figure  6.  Synthetic  image  with  simulated  needle  artifact.  Image  (a)  is  the  sagittal  slice 
acquired  from  the  0.2  T  iMRI  system  without  a  needle  artifact.  Image  (b)  is  obtained  from 
image  (a)  with  a  simulated  needle  artifact  (white  arrow)  for  an  RF  needle  probe  inserted 
into  the  prostate.  Images  are  from  volunteer  S3. 

Figure  7.  Sector  display  showing  quality  of  SV  registration.  Transverse  slices  are  shown 
for  simulated  iMRI  (a)  and  high-resolution  MRI  (b)  images.  In  the  sector  display  (c),  a 
checker  board  pattern  is  created  where  image  sections  from  (a)  and  (b)  are  alternated. 
Square  sections  from  (a)  are  made  brighter  in  order  to  show  the  boundaries.  As  indicated 
by  the  arrows,  the  boundaries  of  bones  and  other  structures  are  continuous  across  the 
sections  indicating  excellent  registration.  The  prostate  registered  very  well.  Images  are 
acquired  from  volunteer  S4. 

Figure  8.  Images  after  SV  registration  of  actual  iMRI  slices  from  a  0.2  T  open  MR 
system.  Image  (a)  is  a  transverse  slice  from  a  high-resolution  MR  volume  (1 .5  T  scanner). 
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The  prostate  is  segmented  and  magnified  in  image  (b).  Image  (c)  is  the  actual  iMRI  slice 
(0.2  T  scanner).  Images  (c)  and  (b)  are  displayed  together  in  an  overlay  in  image  (d),  and 
the  white  rectangular  region  is  magnified  in  image  (e).  The  segmented  prostate  boundary 
from  the  high  resolution  MR  image  is  copied  to  the  actual  iMRI  image  where  it  closely 
matches  the  prostate  in  the  actual  iMRI  image  slice  indicating  excellent  registration. 

Figure  9.  SV  registration  using  images  with  different  slice  thickness.  The  error  metric  is 
the  average  voxel  displacement  between  the  SV  and  W  registrations.  Plotted  are  mean 
errors  as  well  as  standard  deviation  from  a  rectangular  VOI  surrounding  the  prostate.  One 
typical  data  sets  of  high-resolution  MRI  volume  and  actual  iMRI  slices  of  volunteer  SI 
are  used  for  the  registration  experiments.  For  each  thickness,  ten  registration  experiments 
were  conducted  using  10  different  simulated  iMRI  transverse  slices  that  intersected  the 
prostate  with  different  distances.  Thick  iMRI  slices  were  obtained  by  averaging  1-10 
iMRI  image  slices. 

Figure  10.  SV  registration  error  and  robustness  for  iMRI  images  in  the  three  standard 
orientations.  In  (a),  registration  error  relative  to  W  registration  is  plotted  as  a  function  of 
image  slice  orientation.  In  (b),  success  rate  is  also  plotted  as  a  function  of  orientation 
where  registration  is  successful  when  the  error  is  <  2.0  mm.  For  volunteer  S2,  one  high- 
resolution  volume  and  one  volume  from  the  iMRI  scanner  were  used  in  these 
experiments.  Data  were  extracted  from  the  iMRI  volume  to  simulate  iMRI  slices  with  a 
thickness  of  about  5-mm.  Fifteen  transverse,  coronal,  and  sagittal  slices  from  the  prostate 
center  were  used  for  SV  registration,  respectively. 
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ABSTRACT 

The  goal  of  this  research  is  to  register  real-time  interventional  magnetic  resonance  imaging  (iMRI) 
slice  images  with  a  previously  obtained  high-resolution  MRI  image  volume,  which  in  turn  can  be 
registered  with  functional  images  such  as  those  from  SPECT.  The  immediate  application  is  in 
iMRI-guided  treatment  of  prostate  cancer,  where  additional  images  are  desired  to  improve  tumor 
targeting.  In  this  article,  simulation  experiments  are  performed  to  demonstrate  the  feasibility  of 
slice-to-volume  registration  for  this  application.  We  acquired  3D  volume  images  from  a  1.5-T  MRI 
system  and  simulated  low-field  iMRI  image  slices  by  creating  thick  slices  and  adding  noise.  We 
created  a  slice-to-volume  mutual  information  registration  algorithm  with  special  features  to  improve 
robustness.  Features  included  a  multiresolution  approach,  two  similarity  measures,  and  automatic 
restarting  to  avoid  local  minima.  To  assess  the  quality  of  registration,  we  calculated  3D  displace¬ 
ments  on  a  voxel-by-voxel  basis  over  a  volume  of  interest  between  slice-to-volume  registration  and 
volume-to-volume  registration,  which  was  previously  shown  to  be  quite  accurate.  More  than  800 
registration  experiments  were  performed  on  MR  images  of  three  volunteers.  The  slice-to-volume 
registration  algorithm  was  very  robust  and  accurate  for  transverse  slice  images  covering  the  prostate, 
with  a  registration  error  of  only  0.4  ±  0.2  mm.  Error  was  greater  at  other  slice  orientations  and 
positions.  The  automatic  slice-to-volume  mutual  information  registration  algorithm  is  robust  and 
probably  sufficiently  accurate  to  aid  in  iMRI-guided  treatment  of  prostate  cancer.  Comp  Aid  Surg  7: 
257-267  (2002).  ©2003  Wiley-Liss,  Inc. 


Key  words:  image  registration;  mutual  information;  magnetic  resonance  imaging  (MRI);  interven¬ 
tional  MRI;  prostate  cancer;  minimally  invasive  treatment 


INTRODUCTION 

We  use  a  low-field  open  magnet  system  to  guide 
minimally  invasive  treatments,  including  radiofre¬ 
quency  (RF)  thermal  ablation  of  abdominal  can¬ 
cer.1-3  We  are  currently  investigating  the  incorpo¬ 


ration  of  other  medical  images  for  use  in  live-time 
treatment  planning  and  execution.  Examples  of  im¬ 
ages  for  possible  incorporation  include  high-reso- 
lution  MR  images  from  another  scanner  or  from  a 
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very-long-duration  acquisition  in  the  iMRI  scanner, 
MR  spectroscopy  images,  functional  MR  images, 
MR  angiography  images,  PET  images,  and  SPECT 
images.  In  some  cases,  these  other  images  can  be 
used  to  localize  a  disease  process  such  as  a  tumor; 
in  other  cases,  the  images  can  be  used  to  locate  a 
structure  that  should  be  avoided,  such  as  a  critical 
brain  structure  identified  with  functional  MRI. 
iMRI  is  currently  under  investigation  for  possible 
use  in  the  treatment  of  prostate  cancer.  Because 
MRI  does  not  reliably  show  prostate  tumor,  we 
intend  to  incorporate  nuclear  medicine  or  MR  spec¬ 
troscopy  images  with  an  improved  ability  to  detect 
and  localize  the  tumor.4’5 

To  incorporate  image  data  from  other  sources 
in  a  real-time  iMRI  procedure,  we  intend  to  register 
two-dimensional  (2D)  slice  images  acquired 
quickly  on  the  iMRI  scanner  with  a  previously 
acquired  volume  of  image  data.  An  image  volume 
from  another  modality  can  then  be  registered  with 
the  full  MR  volume.  Thus,  to  incorporate  SPECT  in 
an  iMRI  procedure,  we  will  first  register  the 
SPECT  image  volume  with  an  MR  volume.6  Then, 
when  registering  iMRI  slice  images  to  the  MR 
volume,  they  can  also  be  mapped  to  the  SPECT 
functional  image  data.  If  this  procedure  is  success¬ 
ful,  a  variety  of  potential  visualization  tools  will  be 
available  to  help  the  physician  localize  and  apply 
treatments  appropriately.  The  real-time  iMRI  im¬ 
ages  will  be  used  for  guidance,  and  any  small 
misregistration  errors  can  very  probably  be  men¬ 
tally  corrected  by  the  physician.  To  simplify  and 
improve  the  slice-to- volume  (SV)  registration  step, 
we  intend  to  always  use  MR  images  acquired  with 
similar  pulse  sequences.  In  this  report,  we  investi¬ 
gate  SV  registration  of  MR  images. 

Previous  success  with  registering  one  MR 
prostate  volume  to  another7  encourages  us  to  pur¬ 
sue  this  plan.  We  call  this  process  volume-to-vol- 
ume  or  W  registration.  A  rigid-body  mutual  infor¬ 
mation  registration  method  was  used,  with  some 
features  to  improve  robustness.7  We  carefully  eval¬ 
uated  registration  quality  using  a  variety  of  meth¬ 
ods.  For  volume  pairs  acquired  over  a  short  time- 
span  from  a  supine  subject  with  legs  flat  on  the 
table,  registration  accuracy  for  both  prostate  cen¬ 
troids  (typically  <1  mm)  and  bony  landmarks  (av¬ 
erage  1.6  mm)  was  in  the  order  of  one  voxel  (*»1.4 
mm).  For  volumes  acquired  under  very  different 
conditions,  for  example,  with  legs  flat  or  raised  into 
the  treatment  position,  or  with  and  without  bladder 
or  rectal  filling,  we  obtained  somewhat  larger  pros¬ 
tate  centroid  registration  errors  of  about  3.0  mm. 
From  our  results  with  VV  prostate  registration,  it 


was  decided  that  SV  accuracy  could  be  assessed  by 
comparing  results  to  VV  registration  for  those  vol¬ 
ume  pairs  having  low  W  registration  error. 

SV  registration  is  an  alternative,  fast  ap¬ 
proach  for  including  other  modalities  in  iMRI- 
guided  treatment.  Previously,  image  registration 
was  used  to  combine  preoperative  MRI  and  CT 
volumes  for  iMRI-guided  intranasal  microendos¬ 
copy  where  iMRI  volumes  were  acquired  and  reg¬ 
istered  with  preoperative  data.8  Here,  it  is  antici¬ 
pated  that  the  SV  method  will  be  faster,  because 
image  acquisition  is  faster  and  because  registration 
of  a  slice  should  require  many  fewer  calculations 
for  a  volume.  In  addition,  our  experience  is  that  2D 
slice  acquisitions  are  routinely  acquired  during 
iMRI  interventions,  whereas  volume  acquisitions 
are  only  acquired  infrequently.  In  previously  pub¬ 
lished  reports,  SV  registration  was  mainly  applied 
to  the  brain  for  applications  of  functional  MRI,9*10 
postmortem  pathology  studies,11  and  anatomical 
modeling.12  There  are  no  reports  of  SV  registration 
for  abdominal  organs  or  iMRI  guidance. 

The  application  of  SV  registration  methods  to 
iMRI-guided  treatment  of  prostate  cancer  raises 
several  challenges.  First,  iMRI  images  often  have  a 
lower  signal-to-noise  ratio  (SNR)  than  diagnostic 
MR  images  because  of  the  emphasis  on  fast  imag¬ 
ing  and  because  of  the  typically  Jower  field- 
strength  of  open  iMRI  magnets.  Second,  a  single 
slice,  or  a  few  slices,  provides  many  fewer  struc¬ 
tures  than  an  entire  volume  for  voxel-based  match¬ 
ing.  Third,  the  prostate  can  move  relative  to  the 
pelvic  bones  due  to  changes  in  rectal  and  bladder 
filling13’14  or  changes  in  patient  posture  for  treat¬ 
ment 7  That  is,  alignment  of  the  pelvic  bones 
(prominent  anatomical  features  in  MR  gray-scale 
images)  does  not  necessarily  ensure  that  the  pros¬ 
tate  is  aligned.  Fourth,  the  normal  prostate  is  a 
small  organ;  when  healthy,  it  measures  only  —3.8 
cm  in  its  widest  dimension.15  The  prostate  is  lo¬ 
cated  below  the  much  larger  bladder,  which  can 
change  its  shape  and  size  during  imaging.  Finally, 
times  for  registration  and  algorithm  robustness  are 
of  particular  concern  in  this  application. 

In  this  study,  we  test  the  performance  of  SV 
registration.  In  the  next  section,  we  first  describe  a 
voxel-based  registration  method  with  special  fea¬ 
tures  to  improve  robustness.  This  is  followed  by 
details  of  how  entire  MR  volume  pairs  are  acquired 
on  a  conventional  MR  scanner  and  how  realistic 
iMRI  images  are  simulated.  Later,  results  are  pre¬ 
sented  from  over  800  registration  experiments 
comparing  slice-to-volume  with  volume-to- volume 
registration. 
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MATERIALS  AND  METHODS 

Registration  Algorithm 

We  used  an  algorithm  with  special  features  for  the 
slice-to-volume,  or  SV,  registration.  A  similar  algo¬ 
rithm  was  previously  successfully  applied  to  MR  vol- 
ume-to-volume  registration  of  the  prostate.7  The  al¬ 
gorithm  is  outlined  in  the  following  paragraphs.7-16 

We  used  two  similarity  measures,  mutual  in¬ 
formation  (MI)  and  correlation  coefficient  (CC),  in 
the  registration.  One  image,  R,  is  the  reference,  and 
the  other,  F,  is  floating.  Their  mutual  information 
MUR,  F)  is  given  below.17 

mtt.n-'Zrj.r.n  log^^; 

The  joint  probability,  pRF(r,  f),  and  the  mar¬ 
ginal  probabilities  of  the  reference  image,  pR(r), 
and  the  floating  image,  pjff),  can  be  estimated 
from  the  normalized  joint  intensity  histogram.  The 
correlation  coefficient  CC(R,  F)  is  given  below.18 

CC{R,  F) 

.  2(*(r)  ~  R(r))(F(f)  -  F(f)) 

V2(*(r)  -  RdWWU)  -  F(/))2 

Here,  R(r),  F(f)  denote  the  average  intensi¬ 
ties  of  the  reference  and  floating  images,  respec¬ 
tively,  and  the  summation  includes  all  voxels 
within  the  overlap  region. 

We  compared  the  two  similarity  measures  at 
different  resolutions  to  determine  their  suitability 
for  SV  registration.  At  \  resolution,  we  resampled 
images  so  as  to  give  one-quarter  the  number  of 
voxels  along  each  linear  dimension.  At  full  resolu¬ 
tion,  we  used  the  full  number  of  voxels.  In  Figure 
1,  the  two  similarity  measures  are  plotted  as  a 
function  of  translations.  Two  high-resolution  MR 
volumes  were  registered  as  described  previously,7 
and  values  were  plotted  with  the  origin  as  die 
optimal  transformation.  We  calculated  CC  and  MI 
values  while  moving  the  simulated  iMRI  image 
relative  to  the  high-resolution  MR  image  along  the 
coronal  (anterior-posterior)  axis.  The  simulated 
iMRI  image  was  obtained  as  described  later. 

We  use  a  multiresolution  approach  and  per¬ 
form  registration  from  low  to  high  resolution.  We 
use  CC  at  the  two  lower  resolutions  because  it 
gives  fewer  local  maxima  (Fig.  1)  and  can  be 
calculated  faster  than  MI.  We  use  MI  at  full  reso¬ 
lution  because  the  peaked  similarity  function  gives 


a  more  precise  solution  than  CC  (Fig.  1).  To  avoid 
local  maxima,  we  include  a  restarting  feature  where 
registration  is  restarted  with  randomly  perturbed 
parameters  obtained  from  a  uniform  distribution 
about  the  initial  transformation  values  at  the  current 
resolution  being  used.  The  algorithm  restarts  until 
the  absolute  CC  is  above  an  experimentally  deter¬ 
mined  threshold  or  the  maximum  number  of  re¬ 
starts  is  reached.  Absolute  CC  is  used  rather  than 
MI  because  it  has  a  well-defined  range  between  0 
and  1,  and  because  it  provides  an  independent 
check  of  the  MI  result  at  the  highest  resolution. 

All  important  results  are  recorded  following 
an  optimization  cycle,  including  the  CC  and/or  MI 
values,  the  number  of  restarts,  and  the  transforma¬ 
tion  parameters.  At  the  end  of  processing  at  a  lower 
resolution,  we  always  select  the  transformation  pa¬ 
rameters  having  the  maximum  CC  value.  We  then 
scale  the  translation  parameters  appropriately  and 
assign  the  new  parameters  to  be  initial  values  at  the 
next-higher  resolution.  At  the  highest  resolution, 
MI  rather  than  CC  is  the  similarity  measure,  and  we 
select  the  final  transformation  parameters  to  be 
those  with  the  maximum  MI  value. 

For  registration,  we  use  rigid-body  transfor¬ 
mation  (three  translations  and  three  rotations)  and 
trilinear  interpolation,  as  described  previously.19 
For  optimization,  we  use  the  downhill  simplex 
method  of  Nelder  and  Mead.20  Optimization  of 
similarity  ends  either  when  the  maximum  number 
of  calculations  is  reached  (typically  500)  or  when 
the  fractional  change  in  the  similarity  function  is 
smaller  than  a  tolerance  (typically  0.001).  There  are 
several  preprocessing  details  to  note:  The  input 
MRI  volume  is  a  3D  MR  acquisition  giving  256  X 
256  X  128  nearly  isotropic  voxels  over  a  field  of 
view  covering  the  whole  pelvis.  We  create  isotropic 
voxels  of  about  1.37  mm  on  each  side  using  3D 
linear  interpolation.  IDL  (Interactive  Data  Lan¬ 
guage,  Research  System  Inc.,  Boulder,  CO)  is  used 
as  the  programming  language. 

Typical  parameter  values  are  now  described. 
An  initial  guess  is  used  at  the  lowest  resolution  of 
all  zeros  because  the  patient  is  normally  oriented 
approximately  the  same  way  from  one  scan  to  the 
next.  All  CC  thresholds  are  set  at  0.5,  and  the 
maximum  number  of  restarts  is  set  at  20, 10,  and  5, 
from  low  to  high  resolution,  respectively. 

Image  Acquisition 

High-resolution  MRI  volumes  were  acquired  using 
a  1.5-T  Siemens  MRI  system  (Magnetom  Sym¬ 
phony,  Siemens  Medical  Systems,  Erlangen,  Germa¬ 
ny).  An  eight-element  phased-array  body  coil  was 
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(a)  (b) 


Fig.  1.  Similarity  functions  are  plotted  as  a  function  of  translations  in  the  multiresolution  registration  process.  Two 
high-resolution  MRI  volumes  were  registered.  From  the  optimal  parameters,  we  computed  the  similarity  of  the  simulated  iMRI 
and  MRI  images  as  a  function  of  translations  along  the  coronal  (anterior-posterior)  axis.  MI  is  plotted  in  (a)  and  (c);  CC  is 
plotted  in  (b)  and  (d).  Graphs  (a)  and  (b)  are  at  the  lowest  resolution  where  images  are  down-sampled  by  1/4  along  each  linear 
dimension,  giving  a  distance  between  voxel  centers  of  ~5.5  mm.  A  false  global  maximum  for  MI  occurred  at  +25  voxels. 
Graphs  (c)  and  (d)  are  plots  at  full  resolution.  Images  are  from  volunteer  S2. 


used  to  ensure  coverage  of  the  prostate  with  a  uniform 
sensitivity.  Typically,  two  anterior  and  two  posterior 
elements  were  enabled  for  signal  acquisition. 

Two  different  MR  sequences  were  used.  The 
first  was  a  3D  RF  spoiled  gradient  echo  steady-state 
pulse  sequence  (FLASH)  with  TR/TE/flip  parameters 
of  12/5.0/60,  giving  256  X  256  X  128  voxels  over  a 
330  X  330  X  256-mm  field  of  view  (FOV)  to  yield 
1.3  X  1.3  X  2.0-mm  voxels  oriented  to  give  the 
highest  resolution  for  transverse  slices.  The  acquisi¬ 
tion  time  is  5.6  min.  This  sequence  is  good  for  pelvic 
imaging,  but  is  not  ideal  for  the  prostate.  It  was  used 
to  acquire  volumes  for  volunteer  SI. 

The  second  sequence  was  a  3D  rapid  gradient 
echo  sequence  (PSIF)  designed  to  acquire  the  spin- 
echo  component  of  the  steady-state  response,  rather 
than  the  free  induction  decay.  The  spin-echo  com¬ 


ponent  forms  immediately  prior  to  the  RF  pulse, 
and  is  shifted  toward  the  prior  RF  pulse  through 
appropriate  gradient  waveform  design.  The  se¬ 
quence  with  9.4/5.0/60  (TR/TE/flip)  yields  160  X 
256  X  128  voxels  over  a  219  X  350  X  192-mm 
rectangular  FOV  and  1.4  X  1.4  X  1.5-mm  voxels 
oriented  to  give  the  highest  resolution  for  trans¬ 
verse  slices.  There  is  oversampling  at  31%  in  the 
slice  direction  to  reduce  aliasing  artifacts.  The  ac¬ 
quisition  time  is  4.3  min.  This  sequence  gave  ex¬ 
cellent  image  contrast  for  the  prostate,  and  was 
used  to  acquire  volumes  for  volunteers  S2  and  S3. 

Imaging  Experiments 

We  acquired  high-resolution  MRI  volumes  from 
three  volunteers.  For  each  volunteer,  three  image 
volumes  were  obtained  with  an  imaging  session. 
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Fig.  2.  Simulated  iMRI  images.  Images  on  the  left,  (a),  (c),  and  (e),  are  the  original  high-resolution  MR  images  in  the 
transverse,  coronal,  and  sagittal  planes,  respectively.  Images  on  the  right  are  corresponding  simulated  thick  iMRI  images  with 
SNR  =10.  Images  are  from  volunteer  S2. 


Each  volume  was  acquired  with  compatible  condi¬ 
tions.  Volunteers  laid  supine  with  legs  flat,  similar 
to  the  position  in  routine  MR  scanning.  Between 
volume  acquisitions,  volunteers  got  off  the  MRI 
table,  stretched,  and  walked  around  to  ensure  that 
they  would  assume  a  different  position  when  they 
laid  back  on  the  table.  The  coil  array  was  centered 
on  the  prostate.  All  images  of  a  volunteer  were 
acquired  with  the  same  MRI  acquisition  parame¬ 
ters.  In  total,  there  were  nine  pairs  of  high-resolu¬ 
tion  MRI  volumes  for  registration. 

Simulation  of  iMRI  Slice  Images 

We  used  high-resolution  MRI  volumes  to  simulate 
iMRI  images  by  creating  thick  slices  and  adding 
noise  and  receive-coil  inhomogeneity.  Clinically, 
we  typically  use  an  iMRI  slice  thickness  of  4.0-6.0 
mm.  We  averaged  three  slices  1.4  mm  thick  to 
create  a  4.2-mm-thick  slice. 

,  Noise  was  added  to  the  simulated  iMRI  im¬ 
age.  MR  noise  is  described  by  the  Rician  distribu¬ 
tion,21  but  at  reasonably  high  signal  values  it  is 
accurately  approximated  with  Gaussian  white 
noise.22  We  measured  typical  signal  and  noise  val¬ 
ues  on  our  open  magnet  system  using  a  homoge¬ 
nous  phantom  and  methods  described  else¬ 
where  23>24  Gaussian  noise  was  then  added  to  the 
simulated  iMRI  slice  images  either  to  match  the 
measured  SNR  or  to  give  much  greater  noise  to 
further  stress  registration.  We  report  noise  experi¬ 


ments  using  the  SNR  of  the  simulated  slice  images. 
Figure  2  shows  high-resolution  MRI  and  noisy 
simulated  iMRI  slice  images. 

We  simulated  receive-coil  inhomogeneity 
from  a  belt  coil  used  in  the  clinical  iMRI  acquisi¬ 
tions.  The  coil  is  modeled  as  a  solenoid,  and  the 
magnetic  field  is  highest  at  the  coil  center,  falling 
off  in  the  axial  direction.  According  to  the  Biot- 
Savart  law,25  this  model  also  accounts  for  the  spa¬ 
tial  sensitivity  of  the  coil  to  MR  signal  sources. 

Registration  Experiments 

We  desire  an  iMRI  slice  imaging  method  that  gives 
robust,  accurate  registrations,  and  is  relatively  in¬ 
sensitive  to  acquisition  parameters.  Experiments 
were  perfomed  to  determine  the  dependence  on 
slice  orientation  (transverse,  sagittal,  and  coronal), 
on  slice  position  relative  to  the  prostate  (above, 
centered,  and  below),  and  on  image  noise  from  fast 
imaging  techniques.  For  each  volume  pair,  we  ex¬ 
tracted  data  from  one  volume  and  registered  slice 
images  to  the  other  volume.  Many  different  slices 
were  used  in  experiments.  Slice  images  are  simu¬ 
lated  as  described  above. 

Registration  Evaluation 
Visual  Inspection 

Registration  experiments  were  evaluated  by  visual 
inspection.  We  used  RegViz,  a  program  created  in 
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IDL  in  our  laboratory  with  multiple  visualization 
and  analysis  methods.  First,  we  manually  seg¬ 
mented  prostate  boundaries  in  image  slices  and 
copied  them  to  corresponding  slices.  This  enabled 
visual  determination  of  the  overlap  of  prostate 
boundaries  over  the  entire  volume.  Second,  color 
overlay  displays  were  used  to  evaluate  overlap  of 
structures.  One  image  was  rendered  in  gray  and  the 
other  in  the  “hot  iron”  color  scheme  available  in 
DDL.  To  visualize  potential  differences,  it  was  use¬ 
ful  to  interactively  change  the  contribution  of  each 
image  using  the  transparency  scale.  Third,  we  used 
a  sector  display  that  divided  the  reference  and  reg¬ 
istered  images  into  rectangular  sectors  and  created 
an  output  image  by  alternating  sectors  from  the  two 
input  images.  This  way,  even  subtle  shifts  of  edges 
would  be  clearly  seen. 

Comparison  to  Volume-to-Volume 
Registration  Standard 

Our  standard  evaluation  method  was  to  compare 
SV  and  W  registration.  Because  this  relies  on  W 
registration  accuracy,  we  will  now  review  our  pre¬ 
vious  results.7  For  volume  pairs  acquired  over  a 
short  time-span  from  a  supine  subject  with  legs  flat 
on  the  table,  prostates  were  well  aligned,  and  pros¬ 
tate  centroid  displacements  were  typically  <1  mm. 
The  registration  accuracy,  as  determined  from  dis¬ 
placements  of  pelvic  bony  landmarks,  was  1.6  ± 
0.2  mm.  This  error  might  be  overestimated  because 
it  includes  the  uncertainty  of  locating  the  bony 
landmarks.  The  centroid  error  was  slightly  smaller 
because  the  prostate  was  at  the  volume  center  and 
rotation  errors  had  less  effect.  From  our  success 
with  W  prostate  registration,  it  was  decided  that 
we  could  obtain  SV  accuracy  by  comparison  with 
W  registrations  for  those  volume  pairs  having  low 
W  registration  error. 

To  compare  SV  and  W  registration,  we 
defined  a  rectangular  volume  of  interest  (VOI) 
that  just  ^covered  the  prostate  and  calculated 
voxel  displacements  between  the  two  registra¬ 
tions.  To  voxels  within  this  VOI,  we  applied  the 
transformations  obtained  by  W  and  SV  regis¬ 
trations.  We  then  calculated  the  3D  displace¬ 
ments  between  the  transformed  voxels.  The  mean 
voxel  distance  was  used  as  the  metric  of  SV 
registration  error.  For  the  evaluation  of  algorithm 
robustness,  we  defined  the  SV  registration  as 
being  successful  when  the  3D  displacement  was 
less  than  2.0  mm. 


Fig.  3.  Sector  display  showing  excellent  pelvic  registra¬ 
tion.  Image  (a)  is  a  transverse  slice  from  a  high-resolution 
MRI  volume.  Image  (b)  is  the  corresponding  noise  iMRI 
slice.  In  the  sector  display  shown  in  (c),  alternating  rectan¬ 
gular  sections  from  (b)  are  made  brighter  and  combined 
with  sections  from  (a)  to  show  the  matching  boundaries. 
The  boundaries  of  bones  and  other  structures  are  continu¬ 
ous,  as  shown  particularly  at  locations  1-4.  Other  slices 
from  this  volume  were  also  perfectly  aligned,  indicating 
good  3D  alignment.  Image  volumes  are  from  Volunteer  S2. 

RESULTS 
Slice  Orientation 

In  Figure  3,  the  sector  display  shows  a  simulated 
slice  image  registered  with  a  high-resolution  vol¬ 
ume  image.  The  slice  image  was  obtained  at  a 
transverse  orientation  near  the  center  of  the  pros¬ 
tate.  The  sector  display  shows  excellent  alignment 
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Transverse  slice  Coronal  slice  Sagittal  slice 


Fig.  4.  SV  registration  using  slices  at  different  orientations.  The  error  metric  is  the  voxel  displacement  between  the  SV  and 
VV  transformations.  Plotted  are  mean  errors,  as  well  as  maxima  and  minima  over  a  rectangular  VOI  surrounding  the  prostate. 
For  each  volunteer,  SI,  S2,  and  S3,  we  registered  three  volume  pairs.  For  each  pair,  five  registration  experiments  were 
conducted  using  five  different  simulated  iMRI  transverse  slices  intersecting  the  prostate.  The  simulated  SNR  was  25. 
Averaging  data  across  all  volunteers  gives  0.4  ±  0.2  mm,  0.5  ±  0.2  mm,  and  2.6  ±1.6  mm  errors  for  transverse,  coronal,  and 
sagittal  slices,  respectively. 


at  this  position.  Other  transverse  images  were  also 
well  aligned,  indicating  that  the  registration  was 
successful  in  three  dimensions. 

Figure  4  shows  results  for  single  slices  oriented 
in  the  three  traditional  orthogonal  directions.  All 
slices  are  near  the  center  of  the  prostate.  Compared  to 
W  transformations,  registration  error  is  smallest  for 
transverse  slices  and  largest  for  sagittal  slices.  Aver¬ 
aging  across  all  transverse  data,  the  error  is  only  0.4  ± 
0.2  mm,  where  the  latter  number  is  a  standard  devi¬ 
ation.  Coronal  slices  also  gave  quite  small  errors  of 
0.5  ±  0.2  mm.  As  shown  in  Figure  5c,  transverse 
slices  work  best  because  they  contain  abundant  ana¬ 
tomical  structures  that  do  not  deform  relative  to  the 
prostate.  That  is,  a  transverse  slice  centered  at  the 
prostate  excludes  the  bladder  that  can  deform  and 
create  an  inconsistent  match  for  registration.  Figure 
5a,b  shows  that  coronal  and  sagittal  views  contain 
large  regions  of  the  bladder  and  rectum  that  can 
deform  with  filling.  The  following  analyses  are  all 
based  on  transverse  slices. 

Slice  Position 

Figure  6  shows  registration  results  for  transverse 
slices  at  different  distances  from  the  prostate  cen¬ 
ter.  Slices  centered  on  the  prostate  produced  the 
best  results,  with  a  displacement  error  always  less 


than  1.0  mm.  The  reason  is  that  slices  centered  at 
the  prostate  include  an  abundance  of  bony  struc¬ 
tures  giving  good  information  for  registration.  They 
also  exclude  portions  of  the  bladder  that  can  de¬ 
form  and  create  inconsistent  matches  for  registra¬ 
tion  (Fig.  7).  Slices  above  the  prostate  include  the 
deformable  bladder,  which  can  stress  the  registra¬ 
tion  algorithm,  especially  for  volunteer  S3  (Fig.  6). 
Slices  below  the  prostate  mainly  contain  muscle 
and  fatty  regions  from  the  hips  that  can  deform,  and 
there  is  less  information  for  rigid-body  registration. 
This  effect  was  more  pronounced  for  volunteer  S2 
(Fig.  6). 

Noise  Level 

Figure  8  shows  registration  results  for  transverse 
slices  with  added  noise.  The  typical  iMRI  SNR 
under  clinical  conditions  is  about  25.  Even  when 
noise  far  exceeded  this  normal  situation,  regis¬ 
tration  results  were  still  quite  good.  A  100% 
success  rate  was  achieved  with  an  acceptance 
criterion  of  <2.0  mm,  even  when  the  SNR  was  as 
bad  as  10. 

Robustness  and  Calculation  Time 

The  registration  algorithm  was  quite  robust  for 
transverse  slices  covering  the  prostate.  Using  the 
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Fig-.  5.  Image  slices  showing  the  advantage  of  transverse 
images.  The  coronal  slice  in  (a)  contains  the  bladder,  as 
indicated  by  the  black  arrow.  The  sagittal  slice  in  (b)  con¬ 
tains  both  the  bladder  (vertical  arrow)  and  rectum  (horizon¬ 
tal  arrow).  The  transverse  slice  in  (c)  excludes  the  bladder 
and  contains  abundant  bony  structures. 


nine  volume  pairs  from  three  volunteers,  the 
algorithm  never  failed  for  any  transverse  slice 
covering  the  prostate.  In  addition,  the  final  reg¬ 
istration  result  was  insensitive  to  initial  guesses 
within  a  large  range:  [—60,  +60]  mm  for  trans¬ 
lations  and  [—20,  +20]  degrees  for  rotations. 
With  the  restarting  algorithm,  we  even  success¬ 
fully  registered  slices  as  much  as  80  mm  from  the 
optimum.  This  working  range  is  probably  suffi¬ 
cient  for  clinical  applications  where  we  can  en¬ 
sure  good  starting  values.  Using  the  pelvic  bones 
as  markers  and  needle  localization  methods,26  we 
should  be  able  to  position  the  prostate  within 
about  ±20  mm.  In  addition,  the  patient  normally 
lies  supine  in  the  MR  bed  with  very  little  rotation 
(<±5  degrees). 

The  time  for  an  SV  registration  was  typically 
about  5  s  on  a  Pentium  IV  1.8-GHz  CPU  with  1  GB 
of  memory.  The  algorithm  was  written  in  IDL 
and  could  probably  be  made  much  faster  in  a 
lower-level  language  such  as  C.  A  call  to  the  sim¬ 
plex  optimization  typically  resulted  in  50  to  105 
similarity  evaluations  before  the  tolerance  value 
(0.001)  was  reached. 

DISCUSSION 

For  transverse  slices  covering  the  prostate,  the 
slice-to-volume  registration  results  agreed  very  fa¬ 
vorably  with  the  volume-to-volume  results.  That  is. 


Fig.  6.  SV  registration  using  transverse  slices  at  different  positions.  Groups  of  five  iMRI  slices  each  were  extracted  near  the 
prostate  center,  ~35  mm  above  the  prostate  base,  and  -35  mm  below  the  prostate  apex  and  registered  to  the  MR  volume. 
Other  details  are  given  in  Figure  4.  Slices  at  the  prostate  center  worked  best. 
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Fig.  7.  Transverse  images  at  different  positions  relative  to 
the  prostate.  The  image  35  mm  above  the  prostate  in  (a) 
includes  the  bladder,  indicated  by  the  white  arrow.  A  slice 
35  mm  below  the  prostate  in  (b)  mainly  contains  muscle 
(dark)  and  fatty  regions  (white).  A  slice  centered  at  the 
prostate  in  (c)  has  abundant  bony  structures. 


we, found  that  SV  is  equal  to  W  registration,  with 
an  average  voxel  displacement  between  them  of 
only  0.4  mm  in  the  prostate.  The  accuracy  of  SV  is 
essentially  the  same  as  that  previously  reported  for 
W  registration.7  In  our  previous  report,  it  was 
found  that,  whenever  the  subject  was  in  comparable 
conditions,  the  prostate  centroid  registration  error 
was  typically  <1  mm,  or  less  than  a  voxel  of  1.4 
mm.  Hence,  for  SV  applied  to  images  in  this  report, 
we  predict  a  prostate  error  of  very  nearly  1.4  mm  or 
less.  In  the  previous  report,  it  was  found  that  when¬ 


ever  supine  subjects  raised  their  legs,  the  prostate 
moved  towards  the  posterior  direction  by  about  3 
mm.7  Another  factor  that  affects  prostate  registra¬ 
tion  is  rectal  filling.7*13  Hence,  we  recommend  that 
prostate  registration  be  done  with  the  patient  under 
similar  conditions  by  maintaining  a  similar  posture 
and  by  taking  clinical  measures  to  reduce  rectal  and 
bladder  filling.  We  see  no  reason  to  suspect  that  SV 
registration  would  not  be  very  accurate  in  these 
cases. 

Slice-to-volume  registration  is  probably  suf¬ 
ficiently  accurate  for  many  iMRI  applications. 
Compared  to  a  typical  iMRI  slice  thickness  of  —3.0 
mm,  SV  registration  is  quite  accurate.  The  accuracy 
of  SV  is  probably  much  better  than  that  of  multi¬ 
modality  registration,  where  the  typical  functional 
image  has  a  thickness  of  3.0-4.0  mm.  If  one  were 
to  use  functional  or  high-resolution  MR  images 
directly  for  targeting,  the  requirements  for  registra¬ 
tion  accuracy  would  be  great.  However,  fused  im¬ 
age  data  will  not  be  used  blindly.  Rather,  one  can 
use  fused  images  as  a  guide.  Physicians  will  always 
use  the  real-time  anatomical  iMRI  images  for  nee¬ 
dle  guidance.26*27  With  proper  visualization  tools, 
they  should  be  able  to  mentally  account  for  any 
small  registration  errors.  In  addition,  there  is  very 
often  image  evidence  of  cancer  in  MR  prostate 
images  that  can  perhaps  be  identified  with  the  aid 
of  functional  images.  Such  MR-visible  lesions  can 
become  the  markers  for  tumor  targeting.  As  a  result 
of  these  considerations,  we  believe  that  the  accu¬ 
racy  of  SV  registration  is  sufficient  to  justify  fur¬ 
ther  investigation  of  its  application  in  iMRI. 

The  algorithm  is  quite  robust  for  the  SV  reg¬ 
istration.  Significantly,  file  registration  never  failed 
for  transverse  slices  covering  the  prostate.  It  was 
obtained  even  in  the  presence  of  noise  levels  far 
beyond  those  encountered  in  iMRI.  This  is  very 
important  for  iMRI,  where  magnetic  fields  are  low 
and  imaging  is  fast.  It  was  also  determined  that 
coronal  images  work  fairly  well.  This  could  be 
useful,  because  two  orientations  might  provide 
more  flexibility  for  clinical  applications. 

There  are  several  reasons  for  the  robustness. 
First,  using  both  CC  and  MI  at  different  resolutions 
was  an  important  feature  that  increased  robustness. 
When  only  MI  was  used,  registrations  at  low  res¬ 
olution  very  often  gave  false  solutions  that  misled 
registration  at  the  next-highest  resolution.  Fortu¬ 
nately,  CC  performed  well  and  gave  many  fewer 
local  maxima  at  low  resolution.7  However,  MI  gave 
a  more  accurate  solution  at  high  resolution  due  to 
the  peaked  MI  surface.7  Our  registration  algorithm 
combined  advantages  from  the  two  similarity  mea- 
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Fig.  8.  SV  registration  at  different  noise  levels.  The  simulated  iMRI  image  slice  had  SNR  values  of  20,  10,  5,  and  3, 
corresponding  to  added  noise  levels  of  4,  8,  16,  and  27  standard  deviation  gray  levels,  respectively.  The  transverse  slices  from 
Figure  4  were  used.  Other  details  are  given  in  the  legend  of  Figure  4.  Even  though  noise  greatly  exceeds  that  found  in  iMRI, 
registration  was  little  affected. 


sures.  Second,  the  restarting  mechanism  was  also 
quite  important.  Without  restarting,  it  was  found 
that  registrations  sometimes  failed  in  cases  of  vol¬ 
umes  with  a  large  mismatch  of  54  mm  and  high 
noise.  Even  these  cases  resulted  in  a  proper  solution 
when  restarting  was  employed. 

The  simulation  provided  a  realistic,  simple, 
and  efficient  way  to  evaluate  the  algorithm  for  our 
application.  The  simulated  iMRI  images  are  good 
representations  of  images  from  our  low-field- 
strength  iMRI  system.  However,  there  are  other 
practical  aspects  of  iMRI  imaging  that  were  not 
covered  in  the  simulations.  For  instance,  we  did  not 
reproduce  the  susceptibility  artifact  of  an  RF  treat¬ 
ment  needle  in  our  simulations.  From  previous 
experience,19  we  think  this  is  not  a  problem,  be¬ 
cause  the  needle  artifact  occupies  a  relatively  small 
percentage  of  voxels  in  most  cases. 

Another  practical  aspect  is  possible  deforma¬ 
tion  in  the  pelvic  region.  When  images  were  ac¬ 
quired  under  much  different  conditions,  such  as 
supine  or  with  legs  raised,  it  was  determined  that 
warping  was  required  to  successfully  register  the 
prostate.28’29  The  warping  registration  method  was 
based  upon  independent  optimization  of  many  in¬ 
teractively  placed  control  points  using  MI  and  a 
thin-plate  spline  transformation.  About  180  strate¬ 
gically  placed  control  points  were  sufficiently  ex¬ 


pressive  to  capture  important  features  of  the  defor¬ 
mation.  In  the  event  that  a  device  such  as  an 
endorectal  MR  coil  is  used,  warping  registration 
will  be  required. 

Finally,  we  conclude  that  the  slice-to-volume 
registration  algorithm  is  quite  robust  for  transverse 
slice  images  covering  the  prostate,  and  that  regis¬ 
tration  accuracy  is  probably  sufficiently  accurate  to 
aid  iMRI-guided  thermal  ablation  of  prostate  can¬ 
cer.  It  is  quite  feasible  to  include  previously  ac¬ 
quired  high-resolution  MRI  or  nuclear  images  in 
iMRI-guided  treatment  procedures,  and  we  are  be¬ 
ginning  to  explore  this  application  in  animal  exper¬ 
iments. 
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Abstract 

A  three-dimensional  warping  registration  algorithm  was  created  and  compared  to  rigid  body  registration  of  magnetic  resonance  (MR) 
pelvic  volumes  including  the  prostate.  The  rigid  body  registration  method  combines  the  advantages  of  mutual  information  (MI)  and 
correlation  coefficient  at  different  resolutions.  Warping  registration  is  based  upon  independent  optimization  of  many  interactively  placed 
control  points  (CP’s)  using  MI  and  a  thin  plate  spline  transformation.  More  than  100  registration  experiments  with  17  MR  volume  pairs 
determined  the  quality  of  registration  under  conditions  simulating  potential  interventional  MRI-guided  treatments  of  prostate  cancer.  For 
image  pairs  that  stress  rigid  body  registration  (e.g.  supine,  the  diagnostic  position,  and  legs  raised,  the  treatment  position),  both  visual  and 
numerical  evaluation  methods  showed  that  warping  consistently  worked  better  than  rigid  body.  Experiments  showed  that »  180  strategically 
placed  CP’s  were  sufficiently  expressive  to  capture  important  features  of  the  deformation. 
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1.  Introduction 

We  are  investigating  three-dimensional  (3D)  image 
registration,  particularly  with  regard  to  minimally  invasive, 
interventional  magnetic  resonance  imaging  (iMRI)  guided 
treatment  of  prostate  cancer.  At  our  institution,  we  currently 
use  iMRI  on  a  low-field  open  magnet  system  to  guide 
radiofrequency  (RF)  thermal  ablation  of  abdominal  cancer 
[1,2],  and  we  are  investigating  this  method  for  prostate 
cancer  treatment.  A  unique  feature  of  iMRI-guided  thermal 
ablation  is  that  therapy  can  be  monitored  with  MR  either  by 
acquiring  images  of  the  thermally  induced  lesion  or  by 
measuring  temperature.  In  addition,  MR  imaging  of  the 
prostate  is  desirable  because  it  more  accurately  delineates 
the  prostate  than  does  CT  [3],  which  can  overestimate  the 
prostate  volume  [4],  and  ultrasound,  which  has  a  tendency 
to  underestimate  the  extent  of  lesions  [5]. 

Several  applications  in  prostate  cancer  diagnosis, 
staging,  and  therapy  require  registration  of  MR  volumes 
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and/or  volumes  from  other  imaging  modalities.  First, 
registration  of  serial  examinations  can  be  used  to  follow 
regression/progression  of  tumor.  Second,  comparison  of 
registered  MR  images  acquired  before  and  immediately 
after  RF  thermal  ablation  can  be  used  to  determine  whether 
a  tumor  is  adequately  treated.  This  is  particularly  helpful  in 
instances  where  the  edematous  response  to  treatment  can  be 
confused  with  a  highly  perfused  tumor.  Third,  registration 
of  functional,  biochemical  images  such  as  single  photon 
emission  computed  tomography,  positron  emission  tom¬ 
ography  (PET),  and  MR  spectroscopy,  to  anatomical  MR  or 
CT  images  is  useful  for  detecting  and  localizing  cancer  [6]. 
Fourth,  incorporating  the  functional,  biochemical  images 
into  the  iMRI  paradigm  should  aid  image-guided  treatments 
[7].  Finally,  other  treatment  methods  such  as  external 
radiation  therapy,  brachytherapy  [8],  and  surgery,  are  aided 
by  registration  of  images  from  pre-,  intra-,  and  post-therapy 
for  treatment  planning,  guidance,  and  assessment. 

A  few  reports  describe  methods  for  registration  in  the 
pelvis  or  prostate.  Manual  registration  has  been  used  where 
an  operator  cues  on  segmented  vascular  structures  [9]  or 
other  anatomical  landmarks  in  the  pelvis  [6,10-12].  Others 
have  used  automated  3D  schemes  that  match  contours  of 
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bones  and  sometimes  other  structures  that  are  extracted 
using  manual  or  interactive  segmentation  [13-15].  Manual 
segmentation  has  also  been  used  to  create  surfaces  for 
automatic  registration  [16,17],  All  of  these  methods  were 
based  on  rigid  body  registration  and  required  either 
segmentation  or  visual  identification  of  structures. 

We  recently  reported  a  rigid  body  transformation  method 
for  prostate  registration  [18].  For  volume  pairs  acquired 
over  a  short  time  span  from  a  supine  subject  with  legs  flat  on 
the  table,  registration  accuracy  of  both  prostate  centroids 
(typically  <1  mm)  and  bony  landmarks  (average  1.6  mm) 
was  on  the  order  of  a  voxel  («1.4mm).  We  obtained 
somewhat  larger  prostate  registration  errors  of  about 
3.0  mm  when  volume  pairs  were  obtained  under  very 
different  conditions,  e.g.  legs  flat  and  legs  raised,  or  with  and 
without  bladder  or  rectal  filling.  Rigid  body  registration  of 
the  pelvis  cannot  follow  prostate  movements  due  to  changes 
in  the  postures  of  legs  and  deformation  of  the  bladder  and 
rectum,  as  reported  by  us  [18]  and  others  [19,20].  In  this 
report,  we  investigate  the  ability  of  warping  registration  to 
express  this  deformation. 

Warping  registration  studies  are  reported  for  the  brain 
[21,22],  for  the  breast  [23-25],  for  a  variety  of  other  organs 
[26-29],  and  for  excised  tissue  [30].  Far  few  reports 
described  results  of  the  pelvis  and  prostate.  Bharaha  et  al. 
recently  reported  a  method  using  manually  segmented 
prostate  for  rigid  body  registration  followed  by  finite 
element-based  warping  in  the  application  of  prostate 
brachytherapy  [31].  Voxel  based  methods,  particularly 
those  based  upon  mutual  information  (MI),  are  robust, 
require  no  segmentation  that  can  be  prone  to  error,  are 
highly  accurate  for  brain  registration  [32],  and  are  suitable 
for  abdominal  registration  where  there  can  be  deformation 
[2],  We  are  investigating  voxel-based  warping  registration 
for  the  particular  application  in  the  pelvis  and  prostate. 

There  are  challenges  to  pelvis  and  prostate  registration. 
First,  pelvic  regions  can  change  shape  significantly,  unlike 
the  brain  to  which  registration  has  been  most  often  applied. 
Different  patient  positions  such  as  legs  flat  and  raised 
significantly  change  the  legs  in  lower  portions  of  image 
volumes  as  well  as  cause  movement  and  deformation  of 
internal  organs  in  the  pelvis.  Second,  the  normal  prostate  is 
a  small  organ  that  when  healthy  measures  only  about  3.8  cm 
in  its  widest  dimension  transversely  across  the  base  [33]. 
Third,  the  small  prostate  is  located  below  a  much  larger 
bladder  that  can  change  shape  and  size.  Fourth,  the  prostate 
might  move  relative  to  the  pelvic  bones  due  to  changes  in 
bladder  and  rectal  filling  [19,20].  The  alignment  of  the 
pelvic  bones,  a  most  prominent  anatomical  feature  in  MR 
gray-scale  images,  does  not  necessarily  mean  that  the 
prostate  is  aligned.  Finally,  efficacious  application  of 
warping  registration  [29,30]  to  interventional  use  requires 
computational  efficiency. 

In  the  present  study,  we  perform  experiments  to  compare 
warping  and  rigid  body  registration  for  the  prostate  and 
pelvis.  By  using  high-resolution  MR  images  giving 


distinctive  anatomic  detail,  we  test  the  ability  of  a  warping 
algorithm  to  correct  anatomical  variations  throughout  the 
pelvic  region.  We  include  conditions  with  very  significant 
changes  in  posture  possible  in  interventional  applications; 
that  is,  we  ttempt  to  register  image  volumes  from  a 
diagnostic  scan  with  legs  flat  to  those  from  a  treatment 
acquisition  with  legs  raised.  We  qualitatively  and  quanti¬ 
tatively  evaluated  registration  results  using  17  volume  pairs 
from  three  volunteers. 


2.  Registration  algorithm 

2.1.  Similarity  measurements 

We  used  two  similarity  measures,  MI  and  correlation 
coefficient  (CC),  in  our  registration.  Suppose  one  volume  R 
is  the  reference ,  and  the  other  F  is  floating.  Their  MI 
MI(tf ,  F)  is  given  below  [34] 

The  joint  probability  pRF(r,f)  and  the  marginal  probabilities 
pR(r)  of  the  reference  image  and  pF(f)  of  the  floating  image, 
can  be  estimated  from  the  normalized  joint  and  marginal 
intensity  histogram,  respectively.  The  CC  CC (R,  F)  is  given 
below  [35]. 

™  ^  __  I(*to  "  R(r))(F(f)  -  F(f)) 

-  - ■  .i  = 

yjxm)  -  R(r))2Z(F(f)  -  F(f))2 

Here  R(r),  F(f)  denote  the  average  intensities  of  the 
reference  and  floating  volumes  and  the  summation  includes 
all  voxels  within  the  overlap  of  both  volumes. 

2.2.  Rigid  body  registration  algorithm  with  special  features 

Prior  to  warping  registration,  we  perform  rigid  body 
registration  using  a  method  with  features  that  make  it 
particularly  robust  for  MR  pelvic  images.  We  previously 
reported  a  similar  method  [18].  We  use  two  similarity 
measures,  MI  and  CC.  We  use  a  multi-resolution  approach. 
At  low  resolution,  we  resample  both  images  at  1/4  or  1/2 
number  of  voxels  along  each  linear  dimension,  respectively. 
We  use  the  CC  at  these  resolutions  because  it  gives  fewer 
local  maximums  than  MI  [7,18]  and  because  it  can  be 
calculated  faster  than  MI.  We  use  MI  at  full  resolution 
because  the  peaked  similarity  function  gives  a  more  precise 
solution  than  CC  [18].  To  avoid  local  maximums,  we  restart 
with  randomly  perturbed  parameters  obtained  from  a 
uniform  distribution  about  the  initial  transformation  values 
at  the  current  resolution.  The  algorithm  restarts  until  the 
absolute  CC  is  above  an  experimentally  determined 
threshold  or  the  maximum  number  of  restarts  is  reached. 
Absolute  CC  is  used  for  the  restart  test  rather  than  MI 
because  CC  has  a  well-defined  range  between  0  and  1, 
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because  CC  provides  an  independent  check  of  the  MI  result, 
and  because  CC  has  fewer  problems  with  local  and  incorrect 
global  maximums  for  registrations  at  low  resolution  far 
from  the  optimum  value  [18]. 

We  record  all  important  results  following  an  optimiz¬ 
ation  cycle  including  the  CC  and/or  MI  values,  the  number 
of  restarts,  and  the  transformation  parameters.  At  the  end  of 
processing  at  a  lower  resolution,  we  always  select  the 
transformation  parameters  having  the  maximum  CC  value. 
We  then  scale  the  translation  parameters  appropriately  and 
assign  the  new  parameters  to  be  initial  values  at  the  next 
higher  resolution.  At  the  highest  resolution,  we  select  the 
final  transformation  parameters  to  be  those  with  the 
maximum  MI  value. 

Other  details  follow.  A  simplex  algorithm  varies  the  six 
rigid  body  transformation  parameters  (three  translations  and 
three  angles)  to  optimize  the  similarity  measures  [36).  We 
use  an  initial  guess  at  the  lowest  resolution  of  all  zeros 
because  the  patient  is  normally  oriented  approximately  the 
same  way  from  one  scan  to  the  next.  We  set  the  CC 
thresholds  at  a  fixed  value  of  0.50,  and  the  maximum 
numbers  of  restarts  at  10,  5,  and  3,  from  low  to  high- 
resolution,  respectively. 

2.3.  Warping  registration  using  optimized  control  points 

Fig.  1  outlines  the  warping  registration  algorithm  that 
includes  three  major  steps:  control  point  selection,  control 
point  optimization,  and  thin  plate  spline  warping.  The 
unchanging  volume  is  the  reference,  and  the  one  to  be 
warped  is  floating. 


The  manual  selection  of  CP’s  is  an  important  step.  We 
used  RegViz,  a  program  written  in  Interactive  Data 
Language  (IDL,  Research  System  Inc.,  Boulder,  CO)  and 
created  in  our  laboratory  for  visualizing  and  analyzing 
image  volumes.  Following  rigid  body  registration,  the 
aligned  two  volumes  are  displayed  in  two  rows  slice-by¬ 
slice.  Images  can  be  transverse,  coronal,  or  sagittal  slices.  It 
is  quite  straightforward  to  find  corresponding  features  at  the 
pelvis,  prostate,  bladder,  and  rectum.  We  normally  select 
control  points  (CP’s)  using  recognizable  organ  features  such 
as  comers  and  intersections  of  edges  because  of  their  unique 
positions.  Corresponding  CP’s  in  the  two  volumes  are 
placed  using  a  cursor,  and  sometimes  they  are  in  different 
image  slices.  The  3D  coordinates  are  automatically  stored  in 
a  file.  Because  of  the  optimization  that  occurs  later,  the 
correspondence  can  be  up  to  15  mm  or  =  10  voxels  in  error. 
Experiences  with  CP  selection  are  described  in  Section  4. 
Typically,  we  used  180  CP’s  for  a  volume  with 
256  X  256  X  140  isotropic  voxels. 

The  next  step  of  the  warping  algorithm  (Fig.  1)  is  the  CP 
optimization.  We  define  a  small  cubic  volume  of  interest 
(VOI)  centered  at  each  CP.  The  VOI  can  be  16, 32, 48  or  64 
voxels  on  a  side.  As  reported  later,  the  selection  of  VOI  size 
depends  on  the  amount  of  warping  required.  A  simplex 
optimization  algorithm  varies  the  x,  y,  and  z  transformation 
parameters  of  the  floating  VOI  until  the  MI  with  the 
reference  VOI  is  optimized.  Each  control  point  is  optimized 
independently  and  the  3D  coordinates  of  the  optimal  CP’s 
are  recorded. 

The  final  major  step  is  to  warp  the  floating  volume  using 
the  corresponding  optimal  CP’s  coordinates  to  establish 


Register  two  volumes  using  rigid  body  registration 

Create  a  new  floating  volume  from  rigid  body  registration  parameters 

Select  N  control  points  (CP)  in  the  reference  and  new  floating  volumes 

Record  3D  coordinates  of  the  CP’s  in  both  volumes 

Initialize  the  size  of  volume  of  interest  (VOI)  centered  at  CP’s 

Bin  the  reference  and  floating  volumes  to  256  gray  levels 

FOR  CP  FROM  0  to  N-l  DO  BEGIN 

Optimize  mutual  information  (MI)  between  the  reference  and  floating  VOI’ s 

1.  Transform  the  floating  VOI  using  three  translation  parameters 

2.  Interpolate  to  get  a  reformatted  VOI 

3.  Calculate  MI  between  the  reference  and  reformatted  VOI’s 

4.  Vary  the  three  translation  parameters 

5.  Repeat  the  above  steps  1-4  until  meeting  function  tolerance  or  maximum  iteration  number 
Record  the  optimized  CP  coordinates  in  the  floating  volume 

END 


Calculate  thin  plate  spline  transformation  using  the  reference  and  optimized  CP’s 
Interpolate  the  floating  volume  and  get  a  warped  volume 


ri!  ,H°WvnT-  °f  th?  W7ing  ref stration  algorithm  Following  rigid  body  registration,  N CP’s  are  selected  in  both  the  reference  and  floating  volumes.  A 
corresnonlw01  “  “.  red  °n  eachco''tro>  P0»><-  Optrmrzatron  is  performed  by  varying  the  x,  y,  and  z  locations  of  the  floating  VOI  until  the  MI  between 
ZS  todepradenUy,  and  then  the  optimized  CP's  are  uW  (o  aaiablisb  *  ttiree-dimensional  thin 
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a  3D  thin-plate  spline  (TPS)  transformation  [37,38].  We 
now  briefly  go  through  the  three  computing  steps  for  the 
TPS  transformation. 

First,  let  Pi  =  (xuyuzO,  P2  =  (x2,y2iz2) .  Pn  = 

(xn*yn>Zn)  be  n  control  points  in  the  image  coordinate  of 
the  reference  volume.  Write  r(j  =  I Pt  -  Pj\  for  the  distance 
between  point  i  and  j.  We  define  matrices 
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where  T  is  the  matrix  transpose  operator  and  O  is  a  4  X  4 
matrix  of  zero. 

Second,  let  Qx  =  (u^vj,^),  Q2  = 

(M2.v2,w2),...,Gn  =  (««»vB,wn)  be  n  corresponding  CP’s 
in  the  image  coordinate  of  the  floating  volume.  We  get 
matrices 
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0)T, 
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and  define  the  vector  W  =  (w1,w2,...,wn)  and  the  coeffi¬ 
cients  au  ax ,  OLy,  and  az  by  the  equation 

L~lY 

=  (W\  a. 

av 

O' 

Third,  use  the  elements  of  L  lY  to  define  a  function 
f{u!,J,W)  everywhere  in  the  entire  volume 

n 

/(«',  v',  wf)  =  ax  +  auu  +  avv  +  aww  +  ]T  W/lP*  -  («,  v,  w)l. 

i=0 

Thus  any  voxel  (uhvhWi)  in  the  floating  volume  is 
transformed  to  a  new  coordinate  and  a  warped 

volume  can  be  obtained  by  trilinear  interpolation. 

Additional  algorithm  details  are  now  described.  For  both 
VOI  optimization  and  rigid  body  registration,  we  use 
trilinear  interpolation.  Optimization  of  similarity  ends  either 
when  the  maximum  number  of  calculations  is  reached 
(typically  500)  or  the  fractional  change  in  the  similarity 
function  is  smaller  than  a  tolerance  (typically  0,001).  We 
use  IDL  as  the  programming  language. 


3.  Experimental  methods 

3.1.  Image  acquisition 

All  MRI  volumes  were  acquired  using  a  1.5  T  Siemens 
MRI  system  (Magnetom  Symphony,  Siemens  Medical 
Systems,  Erlangen,  Germany).  An  8-element  phased  array 
body  coil  was  used  to  ensure  coverage  of  the  prostate  with  a 
uniform  sensitivity.  Typically  two  anterior  and  two 
posterior  elements  were  enabled  for  signal  acquisition.  We 
used  two  different  MR  sequences.  First,  a  3D  FLASH 
sequence  with  TR/TE/flip  parameters  of  12/5.0/60  gave 
256  X  256  X  128  voxels  over  a  330  X  330  X  256  mm3  field 
of  view  (FOV)  to  yield  1.29  X  1.29  X  2.0  mm3  voxels 
oriented  to  give  the  highest  resolution  for  transverse 
slices.  This  sequence  was  good  for  pelvic  imaging  but 
was  not  ideal  for  prostate  visualization  and  it  was  used  for 
volunteer  SI.  Second,  a  3D  PSIF  sequence  with 
9.4/5.0/60  (TR/TE/flip)  yielded  160  X  256  X  128  voxels 
over  a  219  X  350  X  192  mm3  rectangular  FOV  and 
1.4  X  1.4  X  1.5  mm3  voxels  oriented  to  give  the  highest 
resolution  for  transverse  slices.  There  was  over  sampling  at 
31%  in  the  slice  direction  to  reduce  aliasing  artifacts.  The 
second  sequence  gave  excellent  image  contrast  for  the 
prostate  and  its  surroundings  and  it  was  used  for  volunteers 
S2  and  S3. 

3.2.  Imaging  experiments 

We  acquired  3D  MRI  volume  images  from  three 
normal  volunteers  under  a  variety  of  conditions  simulat¬ 
ing  anticipated  conditions  in  diagnostic  and  treatment 
applications.  Before  image  acquisition,  each  volunteer 
drank  water  and  had  a  relatively  full  bladder .  In  the 
diagnostic  position ,  the  subject  laid  supine  throughout 
MR  scanning.  In  the  treatment  position,  the  subject  was 
supine,  and  his  legs  were  supported  at  30-60°  relative  to 
the  horizon  and  separated  in  a  ‘V*  with  an  angle  of  60- 
90°  between  two  legs.  This  is  similar  to  the  lithotomy 
position  used  in  prostate  therapies,  and  it  should  provide 
access  for  needle  insertion  in  brachytherapy  or  RF 
thermal  ablation.  In  some  experiments,  the  subject 
micturated  to  create  an  empty  bladder  prior  to  imaging. 
For  each  subject,  image  volumes  were  typically  obtained 
on  the  same  day  within  a  2h  session.  We  imaged  one 
volunteer  (S3)  a  week  before  the  standard  imaging 
session,  and  we  refer  to  these  volumes  as  diagnosis  1 
week.  Between  volume  acquisitions,  volunteers  got  off 
the  MRI  table,  stretched,  and  walked  around  to  ensure 
that  they  would  assume  a  different  position  when  they 
laid  back  on  the  table.  The  coil  array  was  centered  on 
the  prostate.  All  images  of  a  volunteer  were  acquired 
with  the  same  MRI  acquisition  parameters  so  as  to 
ensure  very  similar  gray  values.  In  total,  there  are  4,  4, 
and  8  volumes  for  volunteer  SI,  S2,  and  S3,  respectively. 
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The  permutation  of  the  volumes  gives  many  possible 
volume  pairs  for  registration  experiments. 

3.3 .  Volumes  for  registration  experiments 

We  registered  17  volume  pairs  under  five  different 
conditions  as  defined  above.  Five  pairs  are  treatment- 
diagnosis] ;  seven  pairs  are/w//  bladder-empty  bladder, ;  two 
pairs  are  diagnosis  1  week-diagnosis ;  and  three  pairs  are 
diagnosis-diagnosis .  For  each  case,  other  conditions  were 
controlled.  For  example,  for  the  case  of  diagnosis  1  week- 
diagnosis,  both  volumes  were  acquired  with  empty  bladder 
and  comparable  conditions.  Rigid  body  and  warping 
registration  were  applied  to  each  of  the  volume  pairs. 
Results  were  evaluated  as  described  next. 

3.4.  Registration  evaluation 

We  used  the  multiple  visualization  features  of  Reg  Viz  to 
visually  evaluate  registration  results.  First,  we  manually 
segmented  prostate  boundaries  in  image  slices  and  copied 
them  to  corresponding  slices  from  the  other  volume.  This 
enabled  visual  determination  of  the  overlap  of  prostate 
boundaries  over  the  entire  volume.  We  applied  the  same 
method  to  evaluate  pelvic  registration.  Second,  color  overlay 
displays  were  used  to  evaluate  overlap  of  structures.  One 
image  was  rendered  in  gray  and  the  other  in  the  ‘hot-iron* 
color  scheme  available  in  IDL.  To  visualize  potential 
differences,  it  was  quite  useful  to  interactively  change  the 
contribution  of  each  image  using  the  transparency  scale. 
Third,  we  used  a  sector  display,  which  divided  the  reference 
and  registered  images  into  rectangular  sectors  and  created  an 
output  image  by  alternating  sectors  from  the  two  input 
images.  Even  subtle  shifts  of  edges  could  be  clearly  seen  [18]. 

Voxel  gray  value  measures  were  calculated  as  indicators 
of  registration  quality.  MI  and  CC  between  registered 
volumes  were  computed.  Since  volumes  to  be  registered 
were  acquired  using  the  same  acquisition  parameters,  high 
absolute  CC  values  were  obtained  when  registration  was 
good  [24].  Because  voxel  intensities  were  comparable,  we 
created  difference  images  and  calculated  statistics  such  as 
the  voxel  mean  and  standard  deviation  following 
registration. 

Finally,  we  used  a  variety  of  tools  in  RegViz  to  evaluate 
registration  quality.  We  used  contour  overlap  and  color 
overlay  to  assess  the  prostate  registration.  We  manually 
segmented  the  prostate  across  all  slices  and  calculated  the 
potential  displacements  of  the  prostate  3D  centroid. 

4.  Results 

4.1.  Effect  of  control  point  selection  on  registration  quality 

In  well  over  100  registration  experiments  using 
different  numbers  and  placement  of  CP’s,  we  investigated 


effects  on  warping  registration  quality.  For  each  of  the 
three  volunteers,  we  selected  one  typical  volume  pair 
from  the  diagnostic-treatment  positions  for  systematic 
experiments.  We  progressively  increased  the  number  of 
CP’s  from  15  to  250.  We  found  that  less  than  120  CP’s 
did  not  produce  good  visual  matching  of  our  high- 
resolution  MR  images  showing  great  anatomical  detail. 
More  than  220  CP’s  did  not  give  significant  improve 
results  but  required  more  time  for  manual  selection  and 
optimization.  When  we  used  «180  CP’s  placed  strate¬ 
gically  using  rules  described  later,  we  obtained  excellent 
results  over  the  entire  pelvis  and  internal  organs.  As  a 
result  of  our  experience,  we  modified  the  registration 
method  to  be  suitable  for  many  CP’s  (Section  5). 

Some  rules  follow  for  strategic  placement  of  CP’s.  For 
registration  of  treatment  and  diagnostic  image  volumes,  most 
CP’s  were  selected  using  transverse  slices  because  they  best 
showed  the  pelvic  displacement  when  moving  the  legs  to  the 
treatment  position  (Fig.  2).  About  25  CP  pairs  were  placed 
near  edge  and  point  features  having  recognizable  correspon¬ 
dence  on  each  of  5-8  transverse  slices  with  a  z  interval  of 
«  8  mm,  covering  the  entire  pelvic  region.  Additionally,  we 
placed  about  25  CP’s  from  sagittal  slices  because  they 


Fig.  2.  Control  point  selection  when  images  are  acquired  in  the  treatment 
and  diagnostic  positions.  Image  (a)  is  from  the  reference  volume  acquired 
in  the  treatment  position  with  legs  raised.  Image  (b)  is  to  be  warped  and  is 
from  the  volume  acquired  in  the  diagnostic  position  with  the  subject  supine 
on  the  table.  Transverse  slices  best  show  the  deformations,  especially  at  the 
legs.  As  described  in  the  text,  CP’s  indicated  by  the  white  dots  are  selected 
around  the  pelvic  surface  and  the  prostate.  Each  control  point  is  located  at 
one  voxel  but  displayed  much  bigger  for  better  visualization.  Volumes  are 
from  volunteer  S2. 
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provided  other  structures  that  can  be  missed  in  the  transverse 
images.  It  was  also  important  to  include  CP’s  from  organs 
other  than  the  prostate  because  they  constrained  warps.  We 
always  placed  CP’s  at  critical  regions  such  as  the  prostate 
center,  pelvic  surface,  bladder  border,  and  rectal  walls. 

For  registration  of  image  volumes  with  full  and  empty 
bladder,  most  CP’s  were  placed  from  sagittal  slices 
because  they  best  showed  the  deformation  of  the  bladder 
and  rectum  (Fig.  3).  About  10-20  CP’s  were  placed  at 
the  borders  of  the  bladder  arid  rectum  on  each  of  8-10 
sagittal  slices  with  an  equal  interval  of  «8  mm,  covering 
the  entire  pelvic  region  including  the  prostate,  bladder, 
and  rectum. 

4.2.  Registration  quality  of  warping  and  rigid  body 
registration 

In  Fig.  4,  we  compare  warping  and  rigid  body 
registration  for  a  typical  volume  pair  in  the  treatment  and 
diagnostic  positions.  Following  warping  registration,  the 
prostate  boundary  overlap  is  excellent  (Fig.  4(e))  and 
probably  within  the  manual  segmentation  error.  Similar 
results  were  obtained  in  other  transverse  slices  throughout 
the  prostate.  The  prostate  3D  centroid  calculated  from 
segmented  images  displaced  by  only  0.6  mm,  or  0.4  voxels, 
following  warping.  Following  rigid  body  registration,  the 
prostate  was  misaligned  with  a  displacement  to  the  posterior 
of  «3.4  mm  when  in  the  treatment  position  (Fig.  4(d)),  as 
previously  reported  by  us  [18].  Using  rigid  body  regis¬ 
tration,  there  is  significant  misalignment  throughout  large 
regions  in  the  pelvis  (Fig.  4(f))  that  is  greatly  reduced  with 
warping  (Fig.  4(g)).  Note  that  warping  even  allows  the  outer 
surfaces  to  match  well.  Other  visualization  methods  such  as 


Fig.  3.  Control  point  selection  when  images  are  acquired  with  a  week 
interval  between  them.  Image  (a)  is  from  the  reference  volume  acquired  1 
week  later  with  an  empty  bladder.  Image  (b)  is  to  be  warped  and  is  from  the 
volume  acquired  earlier  with  a  full  bladder.  Sagittal  slices  best  show  the 
deformations  at  the  bladder  (vertical  arrow)  and  rectum  (horizontal  arrow) 
where  most  CP’s  are  placed.  Volumes  are  from  volunteer  S3. 


two-color  overlays  and  difference  images,  quickly  show 
matching  of  structures  without  segmentation  but  do  not 
reproduce  well  on  a  printed  page. 

We  next  examine  the  effect  of  conditions  such  as 
bladder  and  rectal  filling  that  might  change  from  one 
imaging  session  to  the  next.  In  Fig.  5,  we  compare 
warping  and  rigid  body  registration  for  a  volume  pair 
with  1  week  between  imaging  sessions.  One  volume  is 
with  an  empty  bladder  and  the  other  is  with  a  relatively 
full  bladder.  There  is  also  a  difference  in  rectal  filling. 
Warping  registration  closely  aligns  the  prostate  Fig.  5(e)) 
while  rigid  body  does  not  (Fig.  5(d)).  In  addition,  rigid 
body  registration  does  not  align  the  bladder  and  parts  of 
the  rectum  (Fig.  5(f)).  With  warping,  the  bladder  closely 
matches  the  reference,  and  the  rectum  is  better  aligned 
(Fig.  5(g)).  Other  visualization  methods  showed  excellent 
alignment  of  internal  and  surface  edges.  Difference 
images  show  that  warping  greatly  improves  alignment 
of  internal  structures  as  compared  to  rigid  body 
registration  (Fig.  6).  The  difference  image  following 
rigid  body  registration  shows  bright  regions  indicating 
misalignments  (Fig.  6(d))  that  are  removed  with  warping 
(Fig.  6(e)). 

We  also  examined  volume  pairs  with  both  volumes 
acquired  in  the  diagnostic  position  under  comparable 
conditions.  In  the  current  data  set,  five  volume  pairs  fit 
these  criteria.  In  all  such  cases,  rigid  body  registration 
worked  as  well  as  warping.  There  were  no  noticeable 
deformations  in  the  pelvis,  and  prostate  centroids  typically 
displaced  less  than  1.0  mm  between  the  two  registered 
volumes.  Note  that  this  was  obtained  even  though  subjects 
always  got  up  from  the  table  and  moved  around  before  being 
imaged  again. 

4.3.  Quantitative  evaluation  of  warping  registration 

Fig.  7  shows  the  CC  and  MI  values  between  registered 
volumes.  Warping  increased  CC  and  MI  values  in  every 
case,  and  a  paired  two-tailed  t  test  indicated  a  significant 
effect  of  warping  at  p  <  0.5%.  The  most  significant 
improvement  was  in  the  case  of  treatment-diagnosis  where 
improvements  in  CC  and  MI  were  as  high  as  102.7  and 
87.8%,  respectively. 

Statistics  of  image  differences  following  rigid  body  and 
warping  registration  are  shown  in  Fig.  8.  Warping  reduces 
the  absolute  intensity  difference  between  corresponding 
voxels  (Fig.  8(a)),  and  the  mean  across  all  image  volumes  is 
only  4.2  gray  levels,  a  value  corresponding  to  only  4.7%  of 
the  mean  image  value  of  90.  We  used  the  absolute  intensity 
difference  because  signed  values  canceled  when  averaged 
over  the  entire  image.  The  standard  deviation  of  absolute 
difference  is  also  reduced  (Fig.  8(b)). 

These  quantitative  measures  match  observation  from 
visual  inspection.  For  example,  the  third  pair  of  the  first 
group  (diagnosis-treatment)  in  Fig.  7  and  8  corresponds  the 
images  in  Fig.  4.  After  warping,  registration  greatly 
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Fig.  4.  Comparison  of  warping  and  rigid  body  registration  for  volumes  acquired  in  the  treatment  and  diagnostic  positions.  Image  (a)  is  from  the  reference 
volume  acquired  in  the  treatment  position,  and  the  prostate  is  manually  segmented.  Images  in  the  left  and  right  columns  are  from  the  floating  volume  acquired 
in  the  diagnostic  position  following  rigid  body  and  warping  registration,  respectively.  To  show  potential  mismatch,  the  prostate  contour  from  the  reference  in 
(a)  is  copied  to  (b)  and  (c)  and  magnified  as  the  dashed  contours  in  (d)  and  (e).  The  3  mm  movement  of  the  prostate  to  the  posterior  is  corrected  with  warping  (e) 
but  not  rigid  body  registration  (d).  Pelvic  boundaries  manually  segmented  from  the  reference  show  significant  misalignment  with  rigid  body  (f)  that  is  greatly 
improved  with  warping  (g).  Images  are  transverse  slices  from  volunteer  S2. 


improved.  Another  interesting  example  is  the  difference 
images  in  Fig.  6(d)  and  (e)  that  correspond  to  the  last  pair  of 
the  second  group  (full-empty  bladder)  in  Fig.  8.  Once 
again,  the  statistical  measures  reflect  the  great  change  in 
visual  quality. 


4A.  Algorithmic  implementation 

In  rigid  body  registration,  the  multi-resolution 
approach  and  restarting  algorithm  were  important  modi¬ 
fications.  First,  these  two  features  improved  robustness. 
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Fig.  5.  Comparison  of  rigid  body  and  warping  registration  for  volumes  acquired  with  an  interval  of  1  week  between  imaging  sessions.  The  reference  image  (a) 
with  a  manually  segmented  prostate  was  acquired  later  with  an  empty  bladder  (vertical  arrow)  and  partial  rectal  filling  (horizontal  arrow).  Images  in  the  left  and 
right  columns  are  from  the  floating  volume  acquired  earlier  following  rigid  body  and  warping  registration,  respectively.  To  show  potential  mismatch,  contours 
from  the  reference  are  shown  on  images  following  registration,  as  described  in  Fig.  4.  The  full  bladder  in  (d)  has  pushed  the  prostate,  shown  by  the  continuous 
curve,  in  the  caudal  direction.  After  warping,  prostate  contours  match  closely  (e).  The  bladder,  rectum,  and  other  organs  closely  align  following  warping  (g). 
With  rigid  body  (f),  proceeding  from  left  to  right,  the  front  of  the  pelvis,  the  bladder  (arrow),  and  the  rectum  are  all  misaligned.  Images  are  sagittal  slices  from 
volunteer  S3. 


The  algorithm  always  gave  very  nearly  the  same 
transformation  parameters  (<0.01  voxels  and  0.01 
degrees)  for  the  17  volume  pairs  in  this  study  using  a 
wide  variety  of  initial  guesses.  We  also  found  that  MI 
was  more  accurate  than  CC  at  the  highest  resolution 
[18].  Second,  the  multi-resolution  approach  enabled  the 
program  to  get  close  to  the  final  value  quickly  because  of 
the  reduced  number  of  calculations.  That  is,  the  time  for 
reformatting  at  the  lowest  resolution  of  1/4  number  of 
voxels  in  a  linear  dimension  was  0.16  min,  less  than  1/63 
times  that  at  the  highest  resolution,  a  value  nearly  equal 
to  the  1/64  expected  from  the  change  in  the  number  of 
voxels.  In  a  typical  example,  the  number  of  restarts  was 
5,  1,  and  1  for  resolutions  at  1/4,  1/2,  and  the  full 
number  of  voxels  in  a  linear  dimension,  respectively. 
When  we  checked  the  restarts  at  the  resolution  of  1/4 
number  of  voxels,  we  determined  that  none  of  the  five 
restarts  converged  to  the  same  transformation.  It  has  been 


our  experience  that  more  restarts  are  desirable  at  the 
lower  resolutions,  and  the  algorithm  includes  this  feature. 
Each  call  to  the  simplex  optimization  resulted  in  50-100 
MI  evaluations  before  the  tolerance  (0.001)  was  reached. 
In  some  experiments  on  multiple  volumes,  we  reduced 
the  tolerance  value  but  found  little  difference  in 
registration  quality,  probably  because  of  the  restarting 
and  multi-resolution  features.  The  time  for  rigid  body 
registration,  typically  5-10  min  on  a  Pentium  IV, 
1.8  GHz  CPU,  with  1.0  GB  of  memory,  could  possibly 
be  reduced  to  within  1  min  with  optimized  C  code  rather 
than  the  high  level  language  IDL. 

Some  technical  aspects  of  warping  registration  are  of 
interest.  Fig.  9  shows  the  optimization  time  and  MI  values 
between  registered  volumes  as  a  function  of  VOI  size.  The 
optimization  time  for  180  CP’s  increases  roughly  linearly 
with  the  number  of  voxels  within  a  VOI,  about  0.5  min  for 
VOI’s  with  16  voxels  on  one  side  and  30  min  for  VOI’s  with 
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Fig.  6.  Comparison  of  registration  quality  for  rigid  body  and  warping  registration.  The  reference  image  (a)  was  acquired  with  a  relatively  empty  bladder 
(arrow).  Images  (b)  and  (c)  are  from  the  floating  volume  acquired  with  a  full  bladder  following  rigid  body  and  warping  registration,  respectively.  Images  (d) 
and  (e)  are  the  absolute  difference  images  between  the  reference  and  registered  images,  respectively.  Bright  regions  following  rigid  body  indicate 
misalignments  (d)  that  are  removed  with  warping  (e).  Images  (d)  and  (e)  are  displayed  using  the  same  gray-scale  window  and  level  values.  Images  are  coronal 
slices  from  S3  volumes  shown  in  Fig.  5. 


64  voxels  on  a  side.  In  Fig.  9,  the  MI  curve  saturates  at  the 
VOI  size  of  64  voxels  on  a  side  that  means  the  size  of  64 
gave  better  MI  value.  These  curves  are  for  the  case  of 
treatment-diagnosis  for  volunteer  S2.  When  we  examined 
the  cases  of  full-empty  bladder  and  volumes  acquired  over 
1  week  time  interval,  we  found  that  the  VOI  size  of  16 
voxels  on  a  side  worked  best.  Using  the  same  computer 
above,  for  a  volume  with  256  X  256  X  140  voxels  and  180 
CP’s,  the  warping  registration  typically  takes  about  15- 
45  min  depending  on  the  VOI  size. 

We  report  some  details  on  VOI  optimization  for  a  typical 
treatment-diagnosis  volume  pair  from  subject  S2.  Following 
rigid  body  registration,  the  mean  distance  between  the 
manually  selected  reference  and  floating  CP’s  was 
15.5  ±  10.7  mm,  where  the  latter  number  is  the  standard 
deviation.  The  maximum  distance  was  53.2  mm.  After  VOI 
optimization,  the  algorithm  moved  the  floating  CP’s  an 
average  of  9.0  ±  6.5  mm.  This  value  shows  that  one  does 
not  have  to  be  very  careful  in  marking  corresponding  CP’s. 


5.  Discussions  and  conclusions 

5.7.  Applicability  of  warping  registration 

For  MR  images  of  the  pelvis  and  prostate,  warping 
registration  is  desirable  whenever  images  are  acquired  in 
different  positions  or  with  different  conditions  of  bladder 
and  rectal  filling.  Local  deformations  throughout  the  pelvis 
can  be  corrected,  and,  more  importantly,  the  prostate  can  be 
accurately  registered.  However,  when  images  are  acquired 


in  the  same  position  under  comparable  conditions  such  as 
our  case  called  diagnosis -diagnosis,  rigid  body  registration 
worked  satisfactorily  as  previously  reported  by  us  [18]. 
Similarly,  if  one  were  to  reproduce  the  treatment  position 
with  reasonable  accuracy,  we  believe  that  prostate  regis¬ 
tration  would  be  very  good. 

Our  goal  is  to  get  good  matching  throughout  the  entire 
pelvic  region  not  just  at  the  prostate  because  proper 
localization  of  other  organs  is  important  for  interpretation 
of  some  functional  images  and  because  anatomical  spatial 
integrity  is  important  for  treatment  planning.  Hence,  we 
used  high-resolution  MR  images  that  provide  a  very 
stringent  test  for  warping.  Many  anatomical  details  are 
evident,  and  even  a  small  mismatch  can  clearly  be  seen.  As  a 
result,  we  found  that  — 180  CP’s  were  required  to  get 
excellent  quality  registration.  When  we  applied  the  method 
to  register  CT  images  with  PET  images  of  the  lung  having 
much  less  resolution,  many  fewer  points  («50)  were 
required  [39].  With  a  sufficient  number  of  CP’s,  the  TPS 
transformation  excellently  approximated  the  deformations 
of  the  pelvis  and  internal  structures  of  our  MR  images.  Even 
when  we  warped  the  volume  in  the  diagnostic  position  to 
one  in  the  treatment  position,  most  organs  were  closely 
aligned,  despite  very  significant  movements.  The  method 
performed  equally  well  for  correcting  the  deformation  and 
organ  displacement  arising  from  changes  in  bladder  and 
rectal  filling. 

With  our  graphical  user  interface,  interactive  control 
point  selection  is  quite  easy  after  training.  It  usually  took  an 
experienced  user  about  15  min  to  select  180  CP’s.  Based  on 
our  experience,  we  think  that  it  is  possible  to  create  an 
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Diagnosis  ~  Full  Bladder  ~  Diagnosis  Diagnosis 

Treatment  Empty  Bladder  ~  Diagnosis  Diagnosis 

1  Week 


Fig.  7.  Voxel  similarity  measures  for  rigid  body  and  warping  registration.  CC  (a)  and  MI  (b)  following  registration  with  (light  bars)  and  without  (dark  bars) 
warping  are  plotted.  Conditions  described  in  Section  3  are  listed  on  the  x-axis.  Warping  increased  CC  and  MI  in  all  cases.  The  most  significant  increases 
occurred  in  the  case  of  the  treatment-diagnosis  volume  pairs  where  maximum  increases  in  CC  and  MI  are  102.7%  and  87.8%,  respectively.  For  volumes 
acquired  with  in  the  same  diagnostic  position  and  comparable  conditions  (two  right  most  groups),  warping  did  not  have  significant  improvement  over  rigid 
body  method. 
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Diagnosis  ~  Full  Bladder  ~  Diagnosis  Diagnosis  ~ 

Treatment  Empty  Bladder  ~  Diagnosis  Diagnosis 

1  Week 

Fig.  8.  Image  statistics  of  absolute  intensity  difference  images  for  rigid  body  and  warping  registration.  The  mean  (a)  and  standard  deviation  (b)  are  plotted.  See 
the  legend  of  Fig.  7  for  other  details.  Warping  decreased  the  mean  and  standard  deviation  in  each  case,  but  the  most  significant  decreases  occurred  in  the  case 
of  the  treatment-diagnosis  volume  pairs.  After  warping,  the  intensity  averaged  over  all  data  is  4.2  ±  1.9  gray  levels,  a  value  corresponding  to  only  ~4.7%  of 
the  mean  image  value  of  ^90  gray  levels. 


278 


B.  Fei  et  at.  /  Computerized  Medical  Imaging  and  Graphics  27  (2003)  267-281 


60 

50 

40 

30 

20 

10 

0 


I 

O 


Fig.  9.  Optimization  time  and  MI  as  a  function  of  VOI  size.  The  left  vertical 
axis  is  MI,  and  the  right  vertical  axis  is  the  total  VOI’s  optimization  time. 
The  horizontal  axis  is  the  size  of  the  VOI  on  a  side.  In  each  case,  the  VOI  is 
centered  on  the  CP,  but  since  even  numbers  of  voxels  are  used,  the  CP  is 
displaced  consistently  to  the  upper  left  hand  comer  by  one  voxel.  With 
increasing  VOI  size,  time  increases  linearly  with  the  number  of  voxels 
within  the  VOL  The  peak  MI  value  is  at  a  VOI  size  of  64  on  a  side.  A 
treatment-diagnosis  volume  pair  is  used  from  S2  with  180  CP’s. 


registration  quality.  First,  for  routine  evaluation,  a  color 
overlay  is  simple,  fast,  and  intuitive.  To  better  visualize  the 
two  data  sets,  we  interactively  adjust  the  transparency  scale 
of  each  image.  Second,  for  illustration  of  subtle  difference 
along  an  edge,  we  recommend  a  sector  display  because  it 
best  shows  small  shifts.  Third,  for  visual  evaluation  of  a 
specific  organ  such  as  the  prostate,  we  like  to  superimpose 
manually  marked  contours  from  one  image  onto  another  as 
shown  in  Figs.  4  and  5.  This  clearly  shows  any  displacement 
or  deformation  even  in  a  printed  figure.  Fourth,  a  more 
quantitative  approach  is  obtained  by  calculating  the 
displacement  in  millimeters  from  the  3D  centroid  of  a 
segmented  organ  such  as  the  prostate.  Finally,  when  images 
have  comparable  gray  levels,  a  difference  image  can  provide 
a  visual  evaluation  or  a  quantitative  evaluation  from  image 
statistics.  A  downside  with  MR  difference  images  is  that  the 
inhomogeneity  of  the  signal  response  and  interpolation  can 
introduce  artifacts  in  difference  images.  Since  MR  image 
intensity  can  vary  with  different  MR  sequence  parameters 
and  the  signal  response  of  MR  coil,  gray  value  statistic  may 
have  some  limitations  when  image  acquisitions  are  not 
carefully  repeated. 


automatic  or  semiautomatic  method  for  selection  of 
appropriate  CP’s  in  the  pelvis.  For  example,  one  might 
use  a  gray  scale  threshold  to  detect  the  pelvic  outer 
boundaries  and  apply  edge  enhancement  to  extract  feature 
of  internal  structures.  CP’s  would  be  placed  on  such 
structures  automatically.  We  are  investigating  this  and 
other  methods  for  CP  selection  followed  by  automatic 
warping  registration. 

One  way  to  adjust  the  movement  of  CP’s  is  to  change  the 
size  of  the  VOI.  In  the  case  of  treatment-diagnosis  volume 
pairs,  a  large  VOI  size  of  64  on  a  side  worked  better  than 
smaller  ones  because  displacements  were  large,  because 
larger  VOI’s  tend  to  give  a  more  robust  optimization,  and 
because  no  small  local  deformations  were  required. 
However,  a  size  of  16  on  a  side  worked  better  for  the  case 
of  full-empty  bladder  volume  pairs  because  small  VOI’s 
better  capture  the  small,  local  deformations.  VOI’s  with  a 
size  of  64  on  a  side  covered  most  of  the  bladder  and  could 
not  generate  small  local  deformations.  For  volumes  with 
both  large  and  small-scale  deformations,  we  suggest  using 
different  VOI  sizes  for  different  CP’s. 

With  warping  registration,  we  have  to  be  concerned 
about  potential  warping  errors  affecting  the  application  of 
interest.  For  the  prostate,  we  used  3-5  CP’s  near  the 
prostate  center  because  we  desired  to  maintain  the  spatial 
integrity  of  the  organ  and  to  preserve  the  tissue  volume.  We 
placed  many  CP’s  around  the  pelvic  surface  to  produce 
reasonable  warping. 

5.2.  Evaluation  of  warping  registration 

Since  there  is  no  gold  standard  for  warping  registration  of 
anatomical  images,  we  used  a  variety  of  methods  to  evaluate 


5.3 .  Algorithmic  robustness  and  efficiency 

The  rigid  body  algorithm  is  robust  for  a  global 
registration.  Because  of  two  principal  design  features,  the 
algorithm  is  quite  robust  and  accurate  for  volume  pairs 
acquired  in  the  same  positions  and  with  comparable 
conditions  [18].  First,  using  both  CC  and  MI  at  different 
resolutions  was  an  important  feature  that  increased  robust¬ 
ness.  CC  gave  fewer  local  minimums  at  low  resolutions  and 
MI  was  more  accurate  at  high-resolution  [7,18].  Second,  the 
restarting  mechanism  was  also  quite  important.  Without 
restarting,  we  found  that  registrations  sometimes  failed  in 
cases  of  volumes  with  large  mismatches  and  significant 
deformation.  Even  these  cases  resulted  in  a  proper  solution 
when  restarting  was  employed. 

Based  upon  our  initial  experiments  with  interactive  CP 
selection,  we  determined  that  many  CP’s  were  required  for 
good  matching  throughout  the  pelvis.  As  a  result,  we 
designed  algorithm  features  to  be  computationally  efficient 
for  TPS  warping  with  hundreds  of  CP’s.  First,  the 
optimization  of  small  VOI’s  is  very  fast.  Second,  we 
optimized  each  CP  separately  because  the  optimization  of 
three  parameters  ( x ,  y,  and  z)  is  simple  and  fast.  Conversely, 
as  previously  reported  by  others  [26,27],  the  simultaneous 
optimization  of  many  CP’ s  leads  to  a  much  more  complicated 
error  surface  and  local  maximums.  If  one  were  to  use  180 
CP’s  and  optimizes  the  540  free  parameters  simultaneously, 
the  optimization  process  would  become  extraordinarily 
complex.  Third,  we  applied  the  TPS  transformation  once  to 
the  final,  optimal  CP’s;  this  saved  considerable  time.  If  TPS 
was  applied  in  each  iteration,  the  registration  time  would  be 
unacceptable  for  our  application.  If  we  were  to  use  optimized 
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C  code,  the  total  time  for  rigid  body  and  warping  registration 
should  reduce  to  within  5  min. 

5.4.  Applications 

We  discuss  several  points  on  volume  interpolation.  We 
used  trilinear  interpolation  in  the  algorithm  because  it  is 
fast.  The  final  floating  volume  following  rigid  body 
registration  is  important  because  it  is  used  for  CP  selection 
and  warping  registration.  A  high  accurate  interpolation  such 
as  sine  spline  [40]  can  be  applied  to  obtain  this  volume  with 
reduced  interpolation  errors.  For  optimization,  partial 
volume  interpolation  that  was  reported  robust  for  MI- 
based  registration  [34]  is  another  option  for  improvement. 

The  flexibility  introduced  with  manual  selection  of  CP’s 
makes  the  current  software  suitable  for  warping  registration 
in  many  applications  in  addition  to  the  clinical  procedures 
described  in  Section  1.  We  have  successfully  applied  it  to 
human  MR-MR  prostate  images  as  shown  here,  rat  CT-CT 
images,  and  CT-PET  lung  images  [39].  We  believe  that  the 
registration  method  can  be  applied  to  many  organs  other 
than  the  pelvis  and  prostate,  multi-modality  images,  and 
inter-subject  images.  In  addition,  we  think  it  applicable  to  a 
variety  of  animal  experiments  in  which  we  are  involved, 
including  iMRI-guided  thermal  ablation  in  pig  and  rabbit, 
prostate  imaging  studies  in  dog,  and  controlled  drug  release 
studies  in  rat. 

We  conclude  that  our  MI  warping  registration  is  fast  and 
can  be  applied  to  a  variety  of  applications.  For  prostate  and 
pelvic  imaging,  it  works  better  than  rigid  body  registration 
whenever  the  subject  position  or  condition  is  greatly 
changed  between  acquisitions.  It  will  probably  be  a  useful 
tool  for  many  applications  in  prostate  diagnosis,  staging, 
and  therapy. 

6.  Summary 

Many  applications  in  prostate  cancer  management  such 
as  tumor  localization,  possibly  tumor  staging,  tumor 
targeting  during  therapy,  assessment  of  adequate  treatment, 
and  treatment  follow  up,  require  image  registration  of  MRI 
volumes  and/or  volumes  from  other  imaging  modalities. 
With  regard  to  interventional  MRI  guided  RF  thermal 
ablation  for  the  minimally  invasive  treatment  of  prostate 
cancer,  registration  applications  include  the  comparison  of 
registered  MR  images  acquired  before  and  immediately 
after  RF  ablation  to  determine  whether  a  tumor  is 
adequately  treated.  When  images  are  acquired  in  different 
patient  positions  and/or  different  conditions,  the  pelvis, 
prostate,  bladder,  and  rectum  can  deform  and  displace. 
Warping  registration  is  desired  to  correct  for  such 
deformations. 

We  created  a  two-step,  3D  registration  algorithm  using 
MI  and  thin  plate  spline  warping  for  the  prostate  MR 
images.  First,  automatic  rigid  body  registration  was  used  to 


capture  the  global  transformation.  Features  included  a 
multi-resolution  approach,  two  similarity  measures,  and 
automatic  restarting  to  avoid  local  minimums.  Second,  local 
warping  registration  was  applied.  Interactively  placed  CP’s 
were  automatically  optimized  by  maximizing  the  MI  of 
corresponding  voxels  in  small  volumes  of  interest  and  by 
using  a  three  dimensional  thin  plate  spline  to  express  the 
deformation  throughout  the  image  volume.  More  than  100 
registration  experiments  with  17  MR  volume  pairs  deter¬ 
mined  the  quality  of  registration  under  conditions  simulat¬ 
ing  potential  interventional  MRI-guided  treatments  of 
prostate  cancer.  Evaluations  included  visual  inspection; 
voxel  gray  value  measures  such  as  MI,  CC,  and  intensity 
difference;  and  displacement  of  the  centroids  of  segmented 
prostates.  For  image  pairs  that  stress  rigid  body  registration 
(e.g.  supine,  the  diagnostic  position,  versus  legs  raised,  the 
treatment  position),  both  visual  and  numerical  evaluation 
methods  showed  that  warping  consistently  worked  better 
than  rigid  body.  Warping  registration  rectified  the  misalign¬ 
ment  in  the  pelvis  following  rigid  body  registration.  The 
prostate  centroid  displacement  for  a  typical  volume  pair  was 
reduced  from  3.4  to  0.6  mm  when  warping  was  added. 
Experiments  showed  that  «180  strategically  placed  CP’s 
were  sufficiently  expressive  to  capture  important  features  of 
the  deformation.  When  only  120  CP’s  were  used,  warping 
throughout  the  pelvis  was  visually  less  satisfactory  but  the 
prostate  was  aligned  reasonably  well.  For  volume  pairs  with 
images  acquired  in  the  same  position  (diagnosis -diagnosis) 
and  comparable  conditions,  the  rigid  body  method  worked 
sufficiently  well,  and  the  prostate  centroid  displacements 
were  <1.0  mm.  In  conclusion,  the  warping  registration 
method  works  better  than  rigid  body  registration  whenever 
patient  position  or  condition  is  greatly  changed  between 
acquisitions.  It  is  very  computational  efficient  for  hundreds 
of  CP’s  and  can  very  well  approximate  the  deformation  of 
the  pelvis  and  internal  organs.  It  will  probably  be  a  useful 
tool  for  many  applications. 
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Abstract 

A  three-dimensional  (3D)  mutual  information  registration  method  was  created 
and  used  to  register  MRI  volumes  of  the  pelvis  and  prostate.  It  had  special 
features  to  improve  robustness.  First,  it  used  a  multi-resolution  approach 
and  performed  registration  from  low  to  high  resolution.  Second,  it  used  two 
similarity  measures,  correlation  coefficient  at  lower  resolutions  and  mutual 
information  at  full  resolution,  because  of  their  particular  advantages.  Third, 
we  created  a  method  to  avoid  local  minima  by  restarting  the  registration 
with  randomly  perturbed  parameters.  The  criterion  for  restarting  was  a 
correlation  coefficient  below  an  empirically  determined  threshold.  Experiments 
determined  the  accuracy  of  registration  under  conditions  found  in  potential 
applications  in  prostate  cancer  diagnosis,  staging,  treatment  and  interventional 
MRI  (iMRI)  guided  therapies.  Images  were  acquired  in  the  diagnostic  (supine) 
and  treatment  position  (supine  with  legs  raised).  Images  were  also  acquired  as 
a  function  of  bladder  filling  and  the  time  interval  between  imaging  sessions. 
Overall  studies  on  three  patients  and  three  healthy  volunteers,  when  both 
volumes  in  a  pair  were  obtained  in  the  diagnostic  position  under  comparable 
conditions,  bony  landmarks  and  prostate  3D  centroids  were  aligned  within 
1.6  ±  0.2  mm  and  1.4  ±  0.2  mm,  respectively,  values  only  slightly  larger 
than  a  voxel.  Analysis  suggests  that  actual  errors  are  smaller  because  of 
the  uncertainty  in  landmark  localization  and  prostate  segmentation.  Between 
the  diagnostic  and  treatment  positions,  bony  landmarks  continued  to  register 
well,  but  prostate  centroids  moved  towards  the  posterior  2.8-3.4  mm.  Manual 
cropping  to  remove  voxels  in  the  legs  was  necessary  to  register  these  images.  In 
conclusion,  automatic,  rigid  body  registration  is  probably  sufficiently  accurate 
for  many  applications  in  prostate  cancer.  For  potential  iMRI-guided  treatments, 
the  small  prostate  displacement  between  the  diagnostic  and  treatment  positions 
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can  probably  be  avoided  by  acquiring  volumes  in  similar  positions  and  by 
reducing  bladder  and  rectal  volumes. 


1.  Introduction 

We  are  investigating  three-dimensional  (3D)  image  registration  to  be  used  in  applications 
of  prostate  cancer  diagnosis,  staging  and  therapy.  In  particular,  we  are  interested  in 
applications  related  to  the  minimally  invasive  interventional  MRI  (iMRI)  guided  treatment 
of  prostate  cancer.  Our  group  currently  uses  iMRI  on  a  low-field  open  magnet  system  to 
guide  radiofrequency  (RF)  thermal  ablation  of  abdominal  cancer  (Lewin  et  al  1998),  and  we 
are  investigating  this  method  for  prostate  cancer  treatment.  A  unique  feature  of  iMRI-guided 
thermal  ablation  is  that  therapy  can  be  monitored  either  by  acquiring  images  of  the  thermally 
induced  lesion  or  by  measuring  temperature.  In  addition,  MR  imaging  of  the  prostate  is 
desirable  because  it  more  accurately  delineates  the  prostate  than  does  CT  (Milosevic  et  al 
1 998),  which  can  overestimate  the  prostate  volume  (Roach  et  al  1996),  and  ultrasound,  which 
has  a  tendency  to  underestimate  the  extent  of  lesions  (Boni  et  al  1995). 

Several  important  applications  require  registration  of  images  of  the  prostate.  First, 
comparison  of  registered  MR  images  acquired  before  and  immediately  after  RF  ablation 
can  be  used  to  determine  whether  a  tumour  is  adequately  treated.  This  is  particularly 
helpful  in  instances  where  the  edematous  response  to  treatment  can  be  confused  with  a 
highly  perfused  tumour.  Second,  registration  of  serial  examinations  can  be  used  to  follow 
regression/progression  of  tumour.  Third,  registration  of  functional,  biochemical  images  such 
as  single  photon  emission  computed  tomography  (SPECT),  positron  emission  tomography 
(PET)  and  MR  spectroscopy  with  anatomical  MR  images  is  useful  for  detecting  and  localizing 
cancer.  Fourth,  incorporating  the  functional,  biochemical  images  into  the  iMRI  paradigm  will 
aid  image-guided  treatments.  Fifth,  on  a  low-field  magnetic  system  during  iMRI  treatments 
where  fast  imaging  is  important,  it  might  be  highly  desirable  to  register  high-quality  MR  image 
from  a  conventional  MR  scanner  to  the  live-time  iMRI  images  (Fei  et  al  2001).  In  this  study, 
we  investigate  registration  of  high-resolution  MR  volumes.  Multi-modality  image  registration 
results  were  reported  elsewhere  (Fei  et  al  2001,  Lee  et  al  2000, 2001a). 

Many  reports  describe  methods  and  evaluations  for  registration  in  the  brain  (Hill  et  al 
200 1 ) ;  far  fewer  describe  results  for  the  pelvis  or  prostate.  For  example,  manual  registration  has 
been  used  where  an  operator  cues  on  segmented  vascular  structures  (Hamilton  et  al  1999)  or 
other  anatomical  landmarks  (Balter  et  al  1995,  Liehn  et  al  1992,  Narayana  et  al  1997).  Others 
have  used  automated  3D  schemes  that  match  contours  of  bones  and  sometimes  other  structures 
that  are  extracted  using  manual  or  interactive  segmentation  (Antolak  et  al  1998,  Herk  et  al 
1998,  Remeijer  et  al  2000).  Manual  segmentation  has  also  been  used  to  create  surfaces  for 
automatic  registration  (Roeske  et  al  1995,  Scott  etal  1994).  All  of  these  methods  require 
either  segmentation  or  visual  identification  of  structures.  Voxel-based  methods,  particularly 
those  based  upon  mutual  information,  are  robust,  require  no  segmentation  that  can  be  prone 
to  error,  are  suitable  for  multi-modality  registration,  are  highly  accurate  for  brain  registration 
(Maes  etal  1997),  and  are  suitable  for  abdominal  registration  (Carrillo  etal  2000).  There 
are  no  reports  of  using  such  methods  for  pelvis  and/or  prostate  registration.  For  registration 
of  brain  and  other  organs,  registration  accuracy  has  been  assessed  using  fiducial  markers 
(Maurer  etal  1997,  Wang  etal  1996)  and  anatomical  landmarks  (Fitzpatrick  et  al  1998, 
Peters  et  al  2000,  Wilson  et  al  1998). 
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There  are  challenges  for  registration  in  the  pelvis  and  prostate  that  might  reduce 
the  effectiveness  of  automatic  voxel-based  registration.  First,  the  abdomen  has  irregular 
boundaries,  unlike  the  head  to  which  registration  has  been  most  often  applied.  Second,  the 
normal  prostate  is  a  small  organ,  which  when  healthy,  measures  only  about  38.0  mm  in  its 
widest  dimension  transversely  across  the  base  (Gray  1977).  Third,  there  are  potential  factors 
such  as  different  patient  positions,  and  rectal  and  bladder  filling  (Herk  etal  1995)  that  can 
stress  registration.  In  addition,  it  is  more  difficult  to  evaluate  pelvic  and/or  prostate  registration 
because  no  external  markers  are  available. 

In  the  present  study,  we  perform  experiments  to  determine  the  potential  accuracy  of 
registering  prostate  MR  images  using  a  modified  mutual  information  algorithm  that  uses  rigid- 
body  transformations.  High-quality,  3D  MR  image  volumes  from  a  commercially  available 
1.5  T  system  are  used  to  determine  the  best  possible  results.  We  examine  conditions  found 
in  potential  applications  described  previously.  We  develop  and  use  a  variety  of  assessment 
methods  that  include  measuring  displacements  of  bony  landmarks  and  of  the  segmented 
prostate.  One  goal  is  to  obtain  baseline  accuracy  measurements  for  planning  future  applications 
of  registration  in  prostate  cancer  management. 


2.  Methods 

2.1.  Data  acquisition 

All  MRI  volumes  were  acquired  using  a  1.5  T  Siemens  MRI  system  (Magnetom  Symphony, 
Siemens  Medical  Systems,  Erlangen,  Germany).  An  8-element  phased  array  body  coil  was 
used  to  ensure  coverage  of  the  prostate  with  a  uniform  sensitivity.  Typically,  two  anterior 
and  two  posterior  elements  were  enabled  for  signal  acquisition.  We  used  two  different 
MR  sequences.  First,  we  used  a  3D  RF  spoiled  gradient  echo  steady-state  pulse  sequence 
(FLASH)  with  TR/TE/flip  parameters  of  12/5.0/60  which  give  256  x  256  x  128  voxels  over  a 
330  x  330  x  256  mm  field  of  view  (FOV)  to  yield  1.29  x  1.29  x  2.0  mm  voxels  oriented 
to  give  the  highest  resolution  for  transverse  slices.  The  acquisition  time  was  5.63  min.  This 
sequence  was  good  for  pelvic  imaging  but  was  not  ideal  for  the  prostate.  Second,  we  used 
a  3D  rapid  gradient  echo  sequence  (PSIF)  designed  to  acquire  the  spin-echo  component  of 
the  steady-state  response,  rather  than  the  free  induction  decay.  The  spin-echo  component 
was  formed  immediately  prior  to  the  RF  pulse  and  it  was  shifted  towards  the  prior  RF  pulse 
through  appropriate  gradient  waveform  design.  The  sequence  with  9.4/5.0/60  (TR/TE/flip) 
yielded  160  x  256  xl28  voxels  over  a  219  x  350  x  192  mm  rectangular  FOV  and  1.37  x 
1 .37  x  1 .5  mm  voxels  oriented  to  give  the  highest  resolution  for  transverse  slices.  There  was 
over  sampling  at  31%  in  the  slice  direction  to  reduce  aliasing  artifacts.  The  acquisition  time 
was  4.25  min.  Most  often,  we  used  the  second  sequence,  which  gave  excellent  image  contrast 
for  the  prostate  and  its  surroundings. 

2.2.  Image  volumes  for  registration 

We  acquired  3D  MRI  volume  images  from  three  prostate  cancer  patients  and  three  normal 
volunteers  under  four  conditions  simulating  anticipated  situations  in  diagnostic  and  treatment 
applications.  They  are  diagnostic  position ,  treatment  position ,  empty  bladder  and  diagnosis 
1  week.  In  the  diagnostic  position,  the  subject  lay  supine  throughout  MR  scanning.  The 
reference  volume  was  always  obtained  in  the  diagnostic  position.  In  the  treatment  position, 
the  subject  was  supine  and  his  legs  were  supported  at  30°-60°  relative  to  the  horizontal 
position  and  separated  in  a  ‘V’  with  an  angle  of  60°-90°  between  the  legs.  This  is  similar 
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to  the  lithotomy  position  used  in  some  prostate  therapies  and  it  should  provide  access  for 
needle  insertion  in  brachytherapy  or  RF  thermal  ablation.  In  some  experiments,  the  subject 
micturated  to  create  an  empty  bladder  prior  to  imaging.  For  each  subject,  volumes  were 
typically  obtained  within  an  imaging  session  of  1-2  h.  We  imaged  one  subject  (V2)  a  week 
before  the  standard  imaging  session  and  we  refer  to  this  volume  as  diagnosis  1  week.  Between 
volume  acquisitions,  subjects  got  off  the  MRI  table,  stretched  and  walked  around  to  ensure 
that  they  would  assume  a  different  position  on  the  table.  The  coil  array  was  centred  on  the 
prostate.  All  images  of  a  subject  were  acquired  using  the  same  pulse  sequence  and  acquisition 
parameters  so  as  to  ensure  very  similar  grey  values.  In  total,  we  registered  22  volume  pairs 
consisting  of  one  pair  for  each  patient,  six  pairs  for  each  volunteer  and  one  additional  pair  for 
volunteer  V2. 

There  are  several  preprocessing  details.  Isotropic  voxels  are  created  using  3D  linear 
interpolation  or  higher  order  interpolation  methods  (Carrillo  etal  2000).  From  the  top  and 
bottom  of  the  volume,  we  optionally  crop  transverse  slices  that  are  over  35  mm  away  from  the 
prostate  rim.  Cropping  is  done  to  remove  slices  having  reduced  brightness  due  to  sensitivity 
fall  off  from  the  receiver  coils,  artifacts  from  a  small  field  of  view,  displacement  of  the  legs  in 
the  treatment  position,  and/or  bladder  deformation. 


2.5.  Similarity  measurements 


We  used  two  similarity  measures,  mutual  information  (MI)  and  correlation  coefficient  (CC), 
in  our  registration.  Suppose  one  volume  R  is  the  reference ,  and  the  other  F  is  floating.  Their 
mutual  information  MI (R,F)  is  given  below  (Maes  et  al  1997): 


MI(R,F)=£>*F(r,/)log 

r.f 


PRFjr ,  /) 
PR(r)-pF(f) 


The  joint  probability  pRF(ry  f)  and  the  marginal  probabilities  pr(t)  of  the  reference  image 
and  pF(f)  of  the  floating  image,  can  be  estimated  from  the  normalized  joint  and  marginal 
intensity  histogram,  respectively.  The  correlation  coefficient  CC(R,F)  is  given  below  (Press 
etal  1993): 


m  -  F(/)) 

t)  =  1  \ . . .  . 

-  *M)2  £(F(/)  -  F(/))2 

Here  R(r),  F(f )  denote  the  average  intensities  of  the  reference  and  floating  volumes  and  the 
summation  includes  all  voxels  within  the  overlap  of  both  volumes. 


2.4 .  Registration  algorithm  with  special  features 

The  registration  algorithm  includes  special  features  to  improve  the  robustness  for  MR  pelvic 
images.  We  use  a  multi-resolution  approach  and  perform  registration  from  low  to  high 
resolution.  At  low  resolution,  we  resample  both  images  at  1  /4  or  1/2  number  of  voxels  along 
each  linear  dimension,  respectively.  Iterative  optimization  of  the  similarity  is  used  to  vary 
the  six  rigid-body  transformation  parameters  (three  translations  and  three  angles).  We  use  the 
correlation  coefficient  at  the  two  lower  resolutions  because  it  gives  fewer  local  maximums  and 
because  it  can  be  calculated  faster  than  MI.  We  use  MI  at  full  resolution  because  the  peaked 
similarity  function  gives  a  more  precise  solution  than  CC. 

We  created  a  method  to  avoid  local  minima  by  restarting  the  registration  with  randomly 
perturbed  parameters  obtained  from  a  uniform  distribution  about  the  very  first  initial  guess  at 
each  resolution.  The  distribution  was  centred  on  the  initial  guess  because  we  wanted  to  use  the 
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best  solution  from  the  lower  resolution.  The  algorithm  restarts  until  the  absolute  correlation 
coefficient  between  the  reference  and  registered  volumes  is  above  a  threshold  or  the  maximum 
number  of  restarts  is  reached.  The  perturbation  range  is  ±5°  and  ±5  voxels  corresponding 
to  ±27.3,  ±13.7  or  ±6.8  mm  for  resolutions  1/4,  1  /2,  or  full  voxels,  respectively.  Absolute 
CC  is  used  for  the  restart  test  rather  than  MI  because  CC  has  a  well-defined  range  between  0 
and  1,  because  CC  provides  an  independent  check  of  the  MI  result,  and  because,  as  described 
later,  CC  has  fewer  problems  with  local  and  incorrect  global  maximums  for  registrations  at 
low  resolution  far  from  the  optimum  value. 

We  record  all  important  results  following  an  optimization  cycle  including  the  CC  and/or 
MI  values,  the  number  of  restarts  and  the  transformation  parameters.  At  the  end  of  processing 
at  a  lower  resolution,  we  always  select  the  transformation  parameters  having  the  maximum  CC 
value.  We  then  scale  the  translation  parameters  appropriately  and  assign  the  new  parameters 
to  be  initial  values  at  the  next  higher  resolution.  At  the  highest  resolution,  we  select  the  final 
transformation  parameters  to  be  those  with  the  maximum  MI  value. 

There  are  several  implementation  details.  We  used  rigid-body  transformation  (three 
translations  and  three  angles)  and  trilinear  interpolation.  For  optimization,  we  use  the  downhill 
simplex  method  of  Nelder  and  Mead  (1965)  and  the  Powell  method  (1962),  but  we  prefer  the 
former  method  as  described  later.  Optimization  of  alignment  ends  either  when  the  maximum 
number  of  MI  calculations  is  reached  (typically  500)  or  the  fractional  change  in  MI  is  smaller 
than  a  tolerance  (typically  0.001).  For  the  22  volume  pairs  reported  here,  the  maximum 
number  of  calculations  was  reached  once  and  this  was  only  at  the  lowest  resolution.  Our  very 
first  initial  guess  at  the  lowest  resolution  is  all  zeros  for  the  three  displacements  and  three 
angles.  Based  on  our  experience,  we  set  the  CC  thresholds  at  0.65,  0.70  and  0.75,  and  the 
maximum  numbers  of  restarts  at  20, 10  and  5,  from  low  to  high  resolutions,  respectively. 


3.  Evaluation  of  registration 

3.1.  Registration  accuracy  based  on  bony  landmarks 

We  evaluated  registration  of  the  pelvis  by  measuring  the  displacement  of  bony  landmarks 
following  registration.  We  used  six  easily  found  bony  landmarks  consisting  of  two  great 
sciatic  notches,  two  lesser  sciatic  notches,  the  pubic  symphysis,  and  the  coccyx,  some  of 
which  are  illustrated  in  figure  1.  Previously,  sciatic  notches  and  the  pubic  symphysis  were 
used  to  register  CT  and  MRI  images  for  prostate  conformal  radiation  (Kagawa  et  al  1997). 
To  measure  landmark  displacements,  we  used  RegViz ,  a  program  written  in  DDL  (Interactive 
Data  Language,  Research  System  Inc.,  USA)  and  created  in  our  laboratory  for  visualizing 
and  analysing  registered  image  volumes.  We  navigated  transverse,  coronal  and  sagittal  MR 
images  slice-by-slice  to  search  the  landmarks.  The  same  unique  features  such  as  comers  and 
intersections  were  identified  with  a  cursor  on  magnified  images.  A  single  person  repeated  this 
six  times  over  a  few  weeks  and  results  were  averaged  to  give  a  3D  location  for  each  landmark. 
A  radiologist  confirmed  the  landmark  selection.  Following  registration,  we  calculated  the 
root-mean-squared  (RMS)  distance  over  the  six  landmarks  (Wang  etal  1996). 

Although  this  method  provides  an  independent  means  for  evaluating  skeletal  registration 
accuracy,  there  is  error  in  localizing  the  bony  landmarks.  To  determine  the  effect  of  localization 
error,  we  performed  least-squares  point-to-point  registration  (Maurer  et  al  1997)  and  compared 
results  to  MI  registration.  The  rationale  is  that  if  we  could  identify  point  landmarks 
without  error  on  the  bony  pelvis,  point-to-point  registration  would  be  perfect.  Hence, 
any  displacement  left  after  registration  is  introduced  by  localization  error.  We  determined 
the  optimal  transformation  for  matching  the  six  corresponding  landmarks.  Points  were 
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Figure  1.  MR  prostate  image  with  labelled  features  used  to  analyse  registration  error.  This 
transverse  image  is  from  the  reference  volume  of  prostate  cancer  patient  P3.  The  prostate  boundary 
was  manually  segmented  near  the  image  centre.  The  four  vertical  dash  lines  from  left  to  right 
indicate  the  rim  of  the  right  acetabular  socket,  the  right  and  left  rims  of  the  prostate  and  the  rim  of 
the  left  acetabular  socket,  respectively.  The  five  crosses  from  bottom  to  top  indicate  the  coccyx, 
the  prostate  posterior  rim,  the  2D  centroid  automatically  calculated  from  the  segmented  prostate 
area,  the  anterior  rim  and  the  pubic  symphysis.  The  image  also  shows  the  bottom  of  the  bladder, 
the  rectum,  the  pubic  symphysis  and  hip  joints. 


transformed  and  distances  between  corresponding  points  were  determined.  RMS  values 
were  computed  and  compared  to  the  RMS  values  from  MI  registration. 

32.  Registration  consistency 

We  calculated  the  registration  consistency  as  proposed  by  Freeborough  et  al  (1996).  For  each 
of  the  three  volunteers,  we  used  three  volumes:  reference,  diagnosis  and  empty  bladder,  all 
of  which  were  obtained  with  the  subject  in  the  supine  position.  We  call  these  three  volumes 
A,  B  and  C,  respectively.  They  give  three  pairs  of  registrations  (A-B,  B-C  and  C-A)  and 
three  sets  of  transformation  parameters  (Tab,  Tbc,  Tca).  Using  the  transformation  parameters, 
we  transformed  voxel  positions  in  A  to  B,  and  then  to  C,  and  then  back  to  A.  The  distance 
between  the  original  location  and  the  final  position  is  calculated.  Since  this  is  introduced  by 
three  transformations,  we  estimate  the  error  for  a  single  transformation,  by  multiplying  by 
3-1/2  (Freeborough  et  al  1996). 

3.3 .  Voxel  displacements 

To  test  the  dependency  of  registration  on  algorithmic  features  such  as  image  cropping,  one 
can  compare  transformation  parameters.  However,  we  chose  a  more  meaningful  approach 
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that  consisted  of  finding  the  average  displacement  of  voxels  in  a  region-of-interest  (ROI) 
(Carrillo  et  al  2000),  The  3D  distances  between  transformed  voxels  were  calculated  in 
millimetres  and  averaged  over  a  cubic  ROI  just  covering  the  prostate. 

3.4.  Other  evaluation  methods  including  displacement  of  prostate  centroids 

We  used  a  variety  of  other  methods  to  evaluate  the  registration  of  the  pelvis  and  prostate.  First, 
we  measured  potential  displacements  of  the  3D  centroid  of  manually  segmented  prostates. 
Second,  we  used  multiple  visualization  and  analysis  methods  found  in  RegViz  and  MIM™ 
(Zalen  LLC,  Novelty,  OH  44072)  such  as  contour  overlap  and  colour  overlay.  Third,  we 
calculated  the  intensity  difference  between  the  reference  and  registered  volumes  on  a  voxel- 
by-voxel  basis  and  computed  statistics.  Fourth,  we  calculated  the  correlation  coefficient 
between  corresponding  voxels  to  measure  the  quality  of  registration  of  two  MR  volumes 
acquired  with  identical  parameters. 

4.  Results 

4.1.  Comparison  of  mutual  information  and  correlation  coefficient 

In  figure  2,  we  compare  the  two  similarity  measures  at  different  resolutions.  Plotted  are  MI 
and  CC  values  as  a  function  of  translation  along  the  transverse  axis  where  the  origin  is  the 
optimal  transformation.  For  images  at  a  resolution  of  1  /4  voxels  along  a  linear  dimension,  the 
CC  curves  are  much  smoother  than  MI,  which  is  noisy  and  contains  many  local  maximums 
as  shown  in  figure  2(a).  In  addition,  there  is  a  false  global  maximum  in  Figure  2(a)  at  18 
voxels.  At  full  resolution,  figure  2(c)  shows  that  MI  is  much  more  peaked  than  CC,  but  there  is 
high-frequency  noise  in  the  MI  curves  far  from  the  optimum  that  gives  rise  to  local  maximums 
that  must  be  avoided.  From  these  figures,  we  infer  that  CC  is  better  at  low  resolution  and 
that  MI  is  better  at  full  resolution  when  one  is  close  to  the  optimum  value.  As  described  in 
section  2,  our  registration  algorithm  makes  use  of  these  features. 

4.2.  Assessments  of  pelvic  registration 

Following  registration,  we  determined  displacements  between  the  six  bony  landmarks.  For 
each  subject,  there  was  no  consistent  displacement  of  landmarks  in  one  direction  versus 
another.  Hence,  we  measured  3D  distances  and  determined  RMS  values  over  the  six  landmarks. 
Registration  results  are  plotted  in  figure  3.  The  smallest  errors  are  obtained  when  subjects 
are  in  the  diagnostic  position  for  both  imaging  sessions,  labelled  diagnosis-reference.  The 
average  error  across  the  three  patients  and  three  volunteers  is  only  1 .6  ±  0.2  mm.  Consistently 
larger  errors  are  obtained  when  we  compare  volumes  acquired  in  the  treatment  position  with 
those  in  the  reference  position.  Even  though  the  MR  acquisition  technique  used  for  the  patients 
gave  inferior  image  quality  as  compared  to  that  for  the  volunteers,  the  errors  were  small. 

Additional  error  analyses  are  performed  on  the  volunteer  images  to  assess  the  accuracy  of 
point  landmark  localization.  We  used  images  obtained  with  the  rapid-gradient  echo  sequence, 
which  have  improved  the  contrast  between  the  prostate  and  bony  landmarks  and  which  give  us 
more  confidence  in  measurements.  The  isotropic  voxels  are  1.4  mm  on  a  side,  almost  as  large 
as  the  1.7  db  0.5  mm  error  obtained  for  the  volunteer  diagnosis-reference  data.  We  assess  the 
error  in  localizing  the  bony  landmarks  by  performing  point-based  registration  on  4-6  points 
per  volume  pair.  The  RMS  distances  after  registration  averaged  across  the  three  volunteers 
was  1.5  ±  0.2  mm,  very  nearly  the  value  obtained  with  MI  registration.  Hence,  the  ‘error* 
reported  for  MI  is  probably  overestimated  due  to  landmark  location  error.  This  analysis  was 
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Figure  2.  MI  and  CC  similarity  functions  are  plotted  to  show  their  relative  advantages  for 
registration  at  different  resolutions.  Two  high-resolution  MRI  volumes  were  registered  to  obtain 
the  optimal  parameters.  We  then  computed  similarity  values  as  a  function  of  translation  along 
the  transverse  axis.  MI  is  plotted  in  (a)  and  (c);  CC  is  plotted  in  (b)  and  (d).  Graphs  on  the  top 
(a)  and  (b)  are  at  a  resolution  of  1/4  voxels  along  a  linear  dimension,  giving  a  distance  between 
voxel  centres  of  ~5.5  mm.  MI  gives  a  noisy  plot  having  many  local  maximums  and  a  false  global 
maximum  occurs  at  18  voxels.  Graphs  at  the  bottom  are  obtained  at  full  resolution.  MI  has  a 
much  sharper  peak  than  CC,  which  is  relatively  flat.  The  voxel  size  is  1.4  mm.  Images  are  from 
volunteer  V2  in  the  diagnostic  and  reference  conditions. 


prompted  by  ideas  in  a  previous  report  (Maurer  et  al  1997)  that  numerically  demonstrated  the 
relationship  between  point  localization  uncertainty  and  point-based  registration  uncertainty. 

Figure  4  shows  image  intensity  differences  between  reference  and  registered  volumes. 
The  means  are  quite  small  with  8  out  of  1 0  registrations  giving  a  mean  absolute  value  <1.5  grey 
levels,  or  only  1.7%  of  typical  mean  values  of  90  grey  levels  for  these  3D  MR  acquisitions. 
Again,  the  only  consistent  outliers  occur  when  we  compare  the  treatment  position  to  the 
reference.  For  the  case  diagnosis-reference,  extremely  small  image  differences  are  found 
with  VI  giving  0.1  ±  1.6  grey  values,  a  standard  deviation  that  compares  favourably  with  the 
expectation  from  image  noise  alone,  or  1.5  grey  values.  For  this  volume  pair,  the  subtracted 
images  have  very  little  structure  except  at  the  skin  surface,  indicating  excellent  registration 
(not  shown).  We  know  that  3D  alignment  is  achieved  because  all  slices  across  the  entire  pelvis 
are  well  aligned  and  because  rendered  images  show  that  the  prostate  matches  well. 
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-  Reference 

Figure  3.  Registration  error  as  determined  from  bony  landmarks.  RMS  distances  as  well  as 
maximums  and  minimums  are  plotted  that  show  die  spread  of  the  data.  Conditions  along  the 
x-axis  such  as  the  ‘treatment’  position  are  described  in  section  2.  P’s  and  V’s  refer  to  patients  and 
volunteers,  respectively.  Averaging  data  across  all  subjects  for  the  best  case  (diagnosis-reference) 
gives  1.6  ±  0.2  mm.  Averages  are  2.9  ±  0.7  mm  and  2.0  d:  0.1  mm  for  treatment-reference  and 
empty  bladder-reference  respectively. 


-  Reference  1  Week 
•Reference 

Figure  4.  Image  intensity  difference  between  registered  volume  pairs.  Means  and  standard 
deviations  calculated  over  the  entire  volumes  are  plotted.  Other  details  are  given  in  the  legend  of 
figure  3.  Eight  out  of  ten  registrations  have  a  mean  absolute  grey  level  difference  less  than  1.5  grey 
levels.  Average  results  are  0.9  ±  1.8,  1.5  ±  3.9,  and  0.4  ±  2.6  grey  levels,  or  1.0,  1.7  and  0.4% 
of  typical  mean  values  of  90  grey  levels,  for  diagnosis-reference,  treatment-reference  and  empty 
bladder-reference  respectively. 
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Registration  consistency,  as  described  in  section  3,  provides  yet  another  means  of 
evaluating  the  quality  of  registration.  Values  were  0.4, 0.8  and  0.7  mm  for  volunteers  VI ,  V2, 
and  V3,  respectively.  The  average  is  0.6  dt  0.2  mm,  a  value  less  than  half  the  dimension  of  a 
voxel  indicating  excellent  registration  consistency. 


43.  Assessment  of  prostate  registration 

We  determined  the  quality  of  prostate  registration  by  visually  examining  nearly  all  of  the 
roughly  800  registered  image  slices  using  one  or  more  of  the  methods  found  in  Reg  Viz  and 
MIM™.  A  typical  example  for  the  case  of  diagnosis-reference  is  shown  in  figure  5  where  the 
boundary  overlap  is  excellent  and  probably  within  the  manual  segmentation  error.  In  some 
other  cases  such  as  treatment-reference,  small  displacements  of  the  prostate  were  observed. 
In  a  typical  volume  pair,  the  prostate  is  displaced  to  the  posterior  direction  by  ^3.0  mm  when 
the  legs  are  raised.  There  are  no  obvious  displacements  in  other  directions. 

Centroid  vector  displacements  can  also  be  analysed  following  registration.  For  the  case  of 
diagnosis-reference,  centroid  displacements  are  only  1 .4  ±  0.2  mm.  In  the  case  of  treatment- 
reference,  there  is  a  consistent  displacement  («3  mm)  in  the  posterior  direction  with  relatively 
little  change  in  the  two  orthogonal  directions.  In  the  case  of  empty  bladder-reference,  two 
of  the  three  volume  pairs  show  a  displacement  in  the  posterior  direction  while  the  other  is 
displaced  in  the  anterior  direction.  Finally,  in  the  case  of  a  diagnostic  volume  obtained  1  week 
before  the  reference,  there  was  a  4  mm  displacement  in  the  caudal  direction  due  mostly  to 
changes  in  rectal  and  bladder  filling.  Because  the  3D  centroid  of  the  prostate  averages  over  a 
large  region,  we  believe  these  measurements  to  be  relatively  insensitive  to  segmentation  error. 
Even  so,  we  consider  the  uncertainty  to  be  at  least  1  mm,  and  displacements  less  than  this 
should  be  disregarded.  All  significant  results  above  can  be  visually  confirmed. 

Prostate  volumes  were  measured  for  each  subject.  The  typical  difference  between  volumes 
in  an  imaging  session  was  <1.5%,  indicating  that  segmentation  errors  were  small  and  that 
prostate  volumes  did  not  change.  The  average  prostate  volume  for  the  healthy  volunteers  was 
23.9  ±  3.2  cm3.  Volume  measurements  are  particularly  useful  for  clinicians  when  assessing 
the  response  of  prostate  cancer  treatments  such  as  brachytherapy,  chemo-  or  radiation  therapy. 


4.4.  Effects  of  image  cropping 

In  figure  6,  we  plot  registration  error  as  determined  from  bony  landmarks  with  and  without  the 
cropping  operation  described  in  section  2.  For  the  case  treatment-reference,  cropping  always 
improved  registration  accuracy,  and  for  V3,  error  reduced  greatly  from  12.6  mm  to  3.4  mm. 
For  all  other  cases,  subjects  were  always  in  the  supine  position  with  legs  flat  on  the  table, 
and  there  was  no  consistent  effect  of  cropping.  If  anything,  cropping  tends  to  increase  error 
in  these  cases,  with  an  increase  in  five  of  seven  volume  pairs.  Correlation  coefficient  always 
improves  with  image  cropping. 

Displacements  of  bony  landmarks  might  significantly  overestimate  the  change  near  the 
prostate.  Hence,  as  described  in  section  3,  we  investigated  the  displacement  of  voxels  in  a 
ROI  surrounding  the  prostate  between  registrations  with  and  without  cropping.  For  nine  of  ten 
analysed  volume  pairs,  the  average  voxel  displacement  was  <0.5  mm  indicating  that  prostate 
registration  is  fairly  insensitive  to  cropping.  However,  for  V3  treatment-reference,  a  much 
larger  voxel  displacement  of  7.4  mm  was  obtained  indicating  that  cropping  is  critical  for  this 
volume  pair. 
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Figure  5.  The  prostate  overlap  between  reference  and  registered  images.  Following  registration, 
the  prostate  was  manually  segmented  in  reference  (a)  and  diagnosis  (b)  images.  The  rectangular 
region  in  (b)  is  zoomed  in  (c)  with  both  boundaries  superimposed.  Images  are  from  volunteer  V2. 
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Figure  6.  The  effect  of  image  cropping  on  registration  accuracy.  The  light  and  dark  bars  are  RMS 
distances  between  bony  landmarks  with  and  without  image  cropping,  respectively,  as  defined  in 
section  2.  Conditions  on  the  jc-axis  are  described  in  section  2. 


4.5.  Implementation  issues 

The  algorithm  was  quite  robust  and  always  gave  very  nearly  the  same  transformation 
parameters  (less  than  0.01  voxels  and  0.01°)  for  the  22  volume  pairs  in  this  study  using  a 
wide  variety  of  initial  guesses.  The  restarting  and  multi-resolution  features  are  important  and 
we  report  some  results  for  a  typical  volume  pair  registration.  The  multi-resolution  approach 
enabled  the  program  to  get  close  to  the  final  value  quickly  because  of  the  reduced  number  of 
calculations.  That  is,  the  time  for  reformatting  at  the  lowest  resolution  (1  /4)  was  9.8  s,  which 
was  less  than  1/59  times  that  at  the  highest  resolution,  a  value  nearly  equal  to  1  /64  expected 
from  the  change  in  the  number  of  voxels.  The  number  of  restarts  was  5, 1  and  1  for  resolutions 
at  1/4, 1/2  and  the  full  number  of  voxels.  Each  call  to  the  Simplex  optimization  resulted  in 
55  to  94  MI  evaluations  before  the  tolerance  (0.001)  was  reached.  The  simplex  optimization 
method  worked  about  1 .5-2.0  times  faster  than  the  Powell  method  in  our  implementation.  The 
time  for  registration  using  Simplex,  typically  10  mines  on  a  Pentium  IV,  1.7  GHz  CPU,  with 
1  Gbytes  of  memory,  could  probably  be  greatly  improved  with  optimized  C  code  rather  than 
IDL. 


5.  Discussion 

5.1.  Registration  accuracy 

Our  results  suggest  that  MI  can  be  used  to  accurately  register,  with  an  error  on  the  order  of  a 
voxel,  MR  pelvic  images  obtained  under  similar  conditions.  Because  it  gives  an  independent, 
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true  3D  measurement,  we  like  to  use  the  method  of  point  bony  landmarks  to  assess  accuracy. 
However,  as  argued  in  section  4.2,  the  true  MI  registration  accuracy  might  be  better  than 
our  ability  to  measure  it  with  point  bony  landmarks.  That  is,  following  point  landmark 
registration,  the  distance  between  registered,  corresponding  landmarks  was  on  the  order  of 
that  following  MI  registration.  Very  possibly,  MI  is  more  accurate  than  point  registration 
using  bony  landmarks.  Additional,  independent  evidence  of  excellent  MI  accuracy  comes 
from  the  very  low  error  value  from  the  registration  consistency  measurement  (0.6  ±  0.2  mm). 
Interestingly,  this  is  obtained  even  though  the  interpolation  artifact  present  in  MI  similarity 
surfaces  should  reduce  the  likelihood  of  sub-voxel  accuracy  (Pluim  et  al  2000a).  Our  results 
for  the  pelvis  with  image  volumes  obtained  under  the  same  conditions  compare  favourably 
with  those  for  the  brain,  where  MI  registers  images  very  accurately  giving  errors  as  small  as 
0.7-0.8  mm  for  CT-MR  (West  et  al  1997). 

Visual  and  quantitative  evaluation  of  prostate  organ  movement  showed  good  registration 
even  when  we  acquired  images  under  conditions  that  greatly  stressed  the  ability  to  register  the 
images.  The  small  prostate  displacements  in  our  study  are  consistent  with  earlier  reports  on 
respiration-induced  prostate  movement  of  <1  mm  for  most  patients  in  supine  position  with 
‘quiet’  respiration  (figure  3(b)  of  the  report  by  Malone  et  al  (2000)).  The  difference  between 
the  treatment  and  diagnostic  positions  resulted  in  the  most  consistent  and  largest  displacement 
of  the  prostate.  When  images  were  acquired  in  the  diagnostic  position  1  week  apart,  there 
was  significant  displacement  of  the  prostate  due  to  a  change  in  rectal  filling.  This  is  consistent 
with  previously  reported  results  (Herk  etal  1995,  Tenhaken  et  al  1991),  which  found  rectal 
filling  to  be  a  significant  factor  in  prostate  displacement. 

There  are  ways  of  limiting  the  small  displacements  of  the  prostate.  One  obvious  remedy 
is  to  acquire  images  in  the  same  position.  That  is,  if  we  want  to  register  an  image  with  that 
obtained  in  the  treatment  position,  we  should  obtain  it  in  the  treatment  position.  Although  it 
is  unknown  how  accurately  one  must  repeat  the  treatment  position,  a  device  to  support  and 
constrain  the  legs  is  probably  required.  In  addition,  there  is  a  dependence  of  registration  error 
on  bladder  and  rectum  content.  One  solution  is  use  clinical  preparations  often  employed  to 
void  the  bladder  and  rectum  prior  to  prostate  imaging  or  therapy.  We  anticipate  that  this  might 
even  lessen  prostate  displacements  between  the  diagnostic  and  treatment  positions. 

We  must  consider  our  results  with  regard  to  potential  applications  such  as  those  described 
in  the  section  1.  First,  registered  images  acquired  before  and  immediately  after  treatment  can 
be  used  to  determine  whether  a  tumour  is  adequately  treated.  Second,  serial  examinations  can 
be  registered  to  determine  tumour  progression  or  regression.  Third,  registration  of  functional 
images  from  other  modalities  such  as  nuclear  medicine  or  from  MR  spectroscopy  can  give 
molecular  markers  for  prostate  cancer  (Lee  et  al  2001a,  2001b).  Fourth,  we  want  to  register 
high-quality  MR  images  with  a  few  live-time  interventional  MR  images  to  aid  treatment 
decisions  (Fei  et  al  2001).  Our  results  indicate  that  registering  images  from  the  treatment  and 
diagnostic  positions  can  lead  to  errors  and  potential  steps  are  described  above  to  limit  this 
error.  With  images  acquired  in  the  same  position,  our  results  place  a  lower  limit  on  registration 
error  of  about  1  voxel.  Experiments  are  being  conducted  to  see  if  this  can  be  achieved  with 
interventional  MR  images  (Fei  et  al  2001). 


5.2.  Assessment  of  registration 

We  are  involved  in  a  long-term  effort  to  use  registration  for  detection,  assessment  and  therapy 
of  prostate  cancer.  Hence,  we  have  developed  and  used  several  methods  to  assess  pelvic  and 
prostate  registration.  ( 
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It  is  highly  desirable  to  have  an  automatic  method  for  evaluating  the  quality  of  a  registration 
so  that  a  poor  one  can  be  flagged  before  it  is  used  clinically.  The  correlation  coefficient  would 
be  applicable  whenever  one  uses  MR  images  obtained  with  identical  pulse  sequences.  It 
compares  favourably  with  the  bony  landmark  results.  Registration  consistency  provides  an 
additional  means  to  evaluate  registration  accuracy  that  does  not  rely  on  operator  interaction. 

Other  evaluation  methods  are  applicable  for  clinical  or  research  applications.  RegViz 
and  MIM™  provided  visual  inspection  tools  for  quick  evaluation  of  the  quality  of  registration 
and  potential  prostate  displacement.  Such  methods  can  be  used  to  verify  the  quality  of 
registration  and  possibly  account  for  small  displacements  in  some  applications.  Boundary 
overlays  provide  a  good  means  to  evaluate  organ  deformation  as  well  as  displacement.  Point 
anatomical  landmarks  provide  a  useful,  independent  test;  but  it  is  time  consuming  to  identify 
them;  and  MI  might  be  more  accurate  than  the  point  landmarks.  Centroids  are  obtained 
reliably  because  small  segmentation  errors  are  removed  by  integrating  over  the  entire  prostate 
volume.  Centroids  provide  a  good  means  of  quantifying  prostate  displacements. 


5.3.  Algorithm  with  combined  similarity  measures 

Using  both  CC  and  MI  at  different  resolutions  was  an  important  feature  that  increased 
robustness.  When  only  mutual  information  was  used,  registrations  at  low  resolution  sometimes 
gave  false  solutions  that  misled  registration  at  the  next  higher  resolution.  However,  CC 
performed  well  and  gave  many  fewer  local  maximums  at  the  lower  resolutions  (figure  2(a) 
and  (b)).  But  MI  gave  a  more  accurate  solution  at  the  full  resolution  due  to  the  peaked  MI 
surface  (figure  2(c)  and  2(d)).  Our  registration  algorithm  combined  advantages  from  the  two 
similarity  measures. 

There  are  probably  several  reasons  why  mutual  information  does  not  work  well  at  low 
resolution.  First,  the  similarity  curve  is  noisy  with  periodic  oscillations  from  the  so-called 
interpolation  artifact  (Pluim  et  al  2000a)  that  is  accentuated  at  reduced  resolutions  (Pluim  et  al 
2000b).  This  results  in  the  many  local  maximums  in  figure  2(a)  that  can  trap  the  optimization. 
A  similar  result  was  reported  for  brain  registration  (Lau  et  al  2001,  Maes  et  al  1997).  Second, 
when  images  are  of  low  resolution  and  there  is  only  a  small  region  of  overlap,  the  mutual 
information  function  can  even  contain  incorrect  global  maximmums  (Pluim  et  al  2000b). 
Such  a  result  was  found  in  figure  2(a)  where  the  global  maximum  was  obtained  at  very  large 
displacements  where  the  overlap  was  reduced.  This  occurs  because  MI  is  not  only  a  function 
of  how  well  the  images  match  in  the  overlap,  but  also  by  how  much  information  is  provided 
by  the  two  images  in  the  overlap  (Studholme  et  al  1997). 


5.4.  Computer  implementation 

Accuracy  is  an  important  issue  for  automatic  registration,  but  there  are  others  such  as 
robustness,  speed,  and  requirements  for  operator  interaction.  With  the  multi-resolution  and 
restarting  features,  our  modified  MI  algorithm  is  quite  robust.  For  a  wide  range  of  initial 
guesses,  it  worked  well  for  all  22  volume  pairs  reported  here.  Three  of  the  volume  pairs  were 
from  patients  and  we  are  confident  that  routinely  acquired  clinical  images  will  have  sufficient 
quality  for  registration.  Because  good  starting  values  are  unimportant,  operator  interaction  is 
minimal.  In  one  instance,  cropping  of  the  legs  was  important  for  registering  an  image  volume 
obtained  in  the  treatment  position  with  that  in  the  diagnostic  position.  It  is  not  surprising  that 
legs  in  a  very  different  position  have  to  be  cropped.  Although  this  is  easy  to  do  manually,  we 
can  probably  determine  an  automated  method  if  it  is  deemed  desirable. 
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The  mutual  information  similarity  measure  is  quite  robust.  Even  though  our  images  are 
very  similar,  we  had  less  success  with  some  other  measures  such  as  the  sum  of  the  squared 
image  difference.  An  advantage  of  MI  is  that  it  can  be  used  with  images  from  different 
modalities,  a  feature  that  we  are  starting  to  use. 

6.  Conclusion 

We  have  developed  an  automatic  volume  registration  algorithm  with  special  features  for  the 
pelvis  and  prostate  MR  volumes.  When  both  volumes  in  a  pair  were  obtained  in  the  diagnostic 
position  under  comparable  conditions,  our  internal  registration  measures  showed  accuracy  on 
the  order  of  a  voxel.  We  believe  that  the  MR  image  registration  method  is  sufficiently  accurate 
and  robust  for  a  variety  of  applications  of  interest  in  the  pelvis  and  prostate.  We  are  beginning 
to  explore  these  applications  in  clinical  procedures  and  animal  experiments. 
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ABSTRACT 

We  are  investigating  interventional  MRI  (iMRI)  guided  radiofrequency  thermal  ablation  for  the  minimally  invasive 
treatment  of  prostate  cancer.  Nuclear  medicine  and  MR  spectroscopy  can  detect  and  localize  tumor  in  the  prostate  not 
reliably  seen  in  MR.  We  are  investigating  methods  to  combine  the  advantages  of  functional  images  such  as  SPECT  with 
iMRI-guided  treatments.  Our  concept  is  to  first  register  the  low-resolution  functional  images  with  a  high  resolution  MR 
volume.  Then  by  registering  the  high-resolution  MR  image  with  live-time  iMRI  acquisitions,  we  can,  in  turn,  map  the 
functional  data  and  high-resolution  anatomic  information  to  iMRI  images  for  improved  tumor  targeting.  In  this  study, 
we  registered  noisy,  thick  iMRI  image  slices  with  high-resolution  MR  volumes  and  called  this  slice-to-volume 
registration.  We  investigated  two  similarity  measures,  i.e.,  mutual  information  and  correlation  coefficient,  and  three 
interpolation  methods,  i.e.,  tri-linear,  re-normalized  sine,  and  nearest  neighbor.  To  assess  the  quality  of  registration,  we 
calculated  3D  displacement  on  a  voxel-by-voxel  basis  over  a  volume  of  interest  (VOI)  between  slice-to-volume 
registration  and  volume-to-volume  registration  that  was  previously  shown  to  be  quite  accurate  for  these  image  pairs. 
Over  300  registration  experiments  showed  that  transverse  slice  images  covering  the  prostate  work  best  with  a 
registration  error  of  only  0.4  ±  0.2  mm.  Error  was  greater  at  other  slice  orientations  and  positions.  Since  live-time  iMRI 
images  are  used  for  guidance  and  registered  images  are  used  for  adjunctive  information,  the  accuracy  and  robustness  of 
slice-to-volume  registration  is  very  probably  adequate. 

Keywords:  Image  registration,  medical  imaging,  interventional  magnetic  resonance  imaging  (iMRI),  mutual 
information,  thermal  ablation,  prostate  cancer. 


1.  INTRODUCTION 

We  are  investigating  interventional  MRI  (iMRI)  guided  radiofrequency  (RF)  thermal  ablation  for  the  minimally 
invasive  treatment  of  prostate  cancer.  Nuclear  medicine  and  MR  spectroscopy  can  detect  and  localize  tumor  in  the 
prostate  not  reliably  seen  in  MR.  We  are  investigating  methods  to  combine  the  advantages  of  functional  images  such  as 
SPECT  with  iMRI-guided  treatments.  Our  concept  is  to  first  register  the  low-resolution  functional  images  with  a  high 
resolution  MRI.  Then  by  registering  the  high-resolution  MR  volume  with  live-time  iMRI  acquisitions,  we  can,  in  turn, 
map  the  functional  data  and  high-resolution  anatomic  information  to  iMRI  images  to  aid  tumor  targeting. 

Previous  success  with  registering  one  MR  prostate  volume  to  another 1  encourages  us  to  pursue  this  plan.  We  call 
this  volume-to-volume  registration,  or  VV.  We  used  a  rigid  body,  mutual  information  registration  method  with  some 
features  to  improve  robustness.1  We  carefully  evaluated  registration  quality  using  a  variety  of  methods.  For  volume 
pairs  acquired  over  a  short  time  span  from  a  supine  subject  with  legs  flat  on  the  table,  registration  accuracy  of  both 
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prostate  centroids  (typically  <  1  mm)  and  bony  landmarks  (average  1 .6  mm)  was  on  the  order  of  a  voxel  ( «  1 .4  mm). 
For  volumes  acquired  under  very  different  conditions,  e.g.,  legs  flat  and  legs  raised  into  the  treatment  position,  or  with 
and  without  bladder  or  rectal  filling,  we  obtained  somewhat  larger  prostate  centroids  registration  errors  of  about  3.0 
mm. 

In  this  study,  we  are  investigating  methods  to  register  live-time  iMRI  image  slices  with  a  previously  obtained,  high 
resolution  MRI  volume.  We  call  this  slice-to-volume  (SV)  registration.  Because  of  our  success  with  VV  prostate 
registration,  we  can  determine  SV  accuracy  by  comparing  results  to  W  registration  for  volume  pairs  having  low  W 
registration  error. 

The  application  of  SV  registration  methods  to  iMRI-guided  treatment  of  prostate  cancer  raises  several  challenges. 
First,  iMRI  images  often  have  lower  signal  to  noise  ratio  (SNR)  than  diagnostic  MR  images  because  of  the  emphasis  on 
fast  imaging  and  because  of  the  typically  lower  field  strength  of  open  iMRI  magnets.  Second,  a  single  slice,  or  a  few 
slices,  provides  many  fewer  structures  than  an  entire  volume  for  voxel  based  matching.  Third,  the  prostate  can  move 
relative  to  the  pelvic  bones  due  to  changes  in  rectal  and  bladder  filling 2,3  or  changes  in  patient  posture  for  treatment.1 
That  is,  alignment  of  the  pelvic  bones,  prominent  anatomical  features  in  MR  gray-scale  images,  does  not  necessarily 
ensure  that  the  prostate  is  aligned.  Fourth,  the  normal  prostate  is  a  small  organ;  when  healthy,  it  measures  only  «  3.8  cm 
in  its  widest  dimension.4  The  small  prostate  is  located  below  the  much  larger  bladder  that  can  change  its  shape  and  size 
during  imaging.  Finally,  times  for  registration  and  algorithm  robustness  are  of  particular  concern  for  this  application. 

Previously  reported  methods  for  SV  registration  were  mainly  applied  to  the  brain  for  applications  of  functional 
MRI,5’6  postmortem  pathology  studies,7  and  anatomical  modeling.8 

We  previously  reported  an  SV  registration  method  for  the  application  of  iMRI-guided  treatment  of  the  prostate.9"12 
In  this  report,  we  further  investigated  the  algorithms  by  comparing  three  interpolation  methods  and  two  similarity 
measures.  We  registered  noisy,  thick  iMRI  image  slices  with  high-resolution  MR  volumes.  Over  300  registration 
experiments  were  performed  with  9  pairs  of  MR  volume  images  acquired  from  three  health  volunteers. 


2.  REGISTRATION  ALGORITHM 


2.1.  Similarity  Measurements 

We  used  two  similarity  measures,  mutual  information  and  correlation  coefficient  (CC),  in  our  registration.  One  image  R 
is  the  reference ,  and  the  other  F  is  floating .  Their  mutual  information  MI(R,  F)  is  given  below.13 

M(R,F)  =  X/v(r,/)i°g  Prff),s, 

r,f  Ps(r)-pF(f) 

The  joint  probability  pRF(r,f)  and  the  marginal  probabilities  pR  (r)  of  the  reference  image  and  pF  (/)  of  the 
floating  image,  can  be  estimated  from  the  normalized  joint  intensity  histograms.  When  two  images  are  geometrically 
aligned,  MI  is  maximal.13  The  correlation  coefficient  CC(R,F)  is  given  below.14 

™  X (tf(r)-*(r))(F(/)-F(/)) 

CC(JV,.T  )  —  ~ jr~r  = _  =• 

Here  R(r),  F(f )  denote  the  average  intensities  of  the  reference  and  floating  images  and  the  summation  includes  all 
voxels  within  the  overlap  of  both  images. 
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2.2.  Interpolation  Methods 


We  used  three  interpolation  methods,  i.e.,  nearest  neighbor  (NN),  tri-linear,  and  re-normalized  sine  interpolation.15,16 
The  conventional  sine  interpolation  with  a  cosine  Hamming  window  is  described  as  below.15 

/'(w)=XEX/(*.  Y,  Z)  ■  HS{x,  X,  R )  •  HS(y,  Y,R )  •  HS(z,  Z,  R) 

X  Y  Z 

with  HS(cc,  A, R)  =  •  {  1  +  cos[7T(a  -  A)/ R  + 1]}, 

2tt  (of  A) 

where  Xf  Y,  Z,  are  the  coordinates  of  the  original  data  set  (/ );  x,  y,  z  are  the  coordinates  of  the  re-formatting  data  set 
(/’ );  A  is  a  symbol  representing  X,  7,  or  Z,  and  a  represents  x,  y,  or  z;  and  R  is  the  kernel  size.  The  Hamming  function 
eliminates  problems  with  oscillatory  effects  at  discontinuities  and  guarantees  that  the  convolution  coefficients  fall  of  to 
zero  at  the  edge  of  the  sine  kernel  (i.e.,  at  /a/=/?+/,  where  R=5  in  this  study).15 

A  previous  report 15  showed  that  re-normalizing  the  interpolation  kernel  for  a  constant  integral  could  make 
significant  improvement  in  performance  of  the  conventional  sine  interpolation.  In  our  implementation,  we  used  the  re¬ 
normalized  sine  interpolation  method.15  We  replaced  HS  in  the  above  equation  with 

HS(a,A,R)  =  HS(oc,  A,  R)j^  HS(a,  A,  R)  =  HS(a,A,R)/\HS(a)\ . 


2.3.  Comparison  of  Different  Similarity  Measures  and  Interpolation  Methods 

We  compared  two  similarity  measures,  CC  and  MI,  at  different  resolutions  in  order  to  determine  their  suitability  for  SV 
registration.  At  1/4  resolution ,  we  resampled  images  so  as  to  give  1/4  number  of  the  voxels  along  each  linear 
dimension.  At  full  resolution,  we  used  the  full  number  of  voxels.  We  plot  the  two  similarity  measures  as  a  function  of 
translations.  After  two  typical  high-resolution  MR  volumes  were  registered,1  values  were  plotted  with  the  origin  as  the 
optimal  transformation.  We  calculated  CC  and  MI  values  while  moving  the  simulated  iMRI  image  relative  to  the  high- 
resolution  MR  image  along  coronal  (anterior-posterior)  axis.  When  obtaining  floating  images,  we  used  the  three 
different  interpolation  methods. 

Features  of  MI  and  CC  demonstrate  their  suitability  at  high  and  low  resolutions,  respectively.  At  1/4  resolution,  CC 
curves  are  much  smoother  than  MI,  which  is  noisy  and  contains  many  local  maxima  as  shown  in  Figures  1.  At  full 
resolution,  Figures  2  shows  that  MI  curves  are  much  peaked  than  CC,  but  once  again  there  is  high  frequency  noise  in 
the  MI  curves,  far  from  the  optimum,  that  gives  rise  to  local  maxima  that  must  be  avoided.  From  these  figures,  we  infer 
that  CC  is  better  at  low  resolution  and  MI  is  better  at  full  resolution,  when  one  is  close  to  the  optimum  value. 

We  also  compared  the  effects  interpolation  methods.  At  1/4  resolution  (Figure  1),  the  MI  curves  with  sine  had  more 
local  maximums  than  those  with  tri-linear;  however,  CC  plots  with  sine  were  smoother  than  tri-linear.  This  indicates 
that  CC  and  sine  are  possibly  the  best  choices  for  robust  registration  at  low  resolution.  At  full  resolution  (Figure  2),  MI 
and  sine  gave  the  sharpest  peak  at  the  optimum;  nearest  neighbor  interpolation  gave  a  flat  peak  with  a  width  of  one 
voxel;  and  tri-linear  gave  a  result  between  the  other  two.  These  results  imply  that  MI  and  sine  interpolation  should 
provide  the  best  registration  accuracy  at  full  resolution.  However,  tri-linear  interpolation  is  about  100  times  faster  than 
sine  interpolation  in  our  implementations. 
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Figure  1.  Similarity  functions  are  plotted  as  a  function  of  translations  at  the  1/4  resolution  in  the  multi-resolution  registration 
process.  Two  high-resolution  MRI  volumes  were  registered,  and  they  are  down  sampled  by  1/4  along  each  linear  dimension,  giving 
a  distance  between  voxel  centers  of  ~  5.5  mm.  From  the  optimal  parameters,  we  computed  the  similarity  of  the  simulated  iMRI  and 
MRI  images  as  a  function  of  translations  along  the  coronal  (anterior-posterior)  axis.  Correlation  coefficient  (CC)  is  plotted  in  the 
left  column.  Mutual  information  (MI)  is  plotted  in  the  right  column.  From  the  top  to  bottom,  nearest  neighbor  (NN),  tri-linear  and 
re-normalized  sine  interpolation  methods  are  used  to  obtain  die  floating  images,  respectively.  The  noisey  MI  curves  in  the  right 
column  show  many  local  maxima,  especially  with  sine  interpolation.  CC  curves  in  the  left  column  are  much  smoother  indicating  its 
suitability  for  low  resolution.  Nearest  neighbor  has  a  flat  peak  with  a  width  of  one  voxel  in  both  similarity  curves.  The  unit  of  the 
X-axis  is  voxel.  Images  are  from  volunteer  S2. 
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Figure  2.  Similarity  functions  are  plotted  as  a  function  of  translations  at  full  resolution.  Many  details  are  given  in  the  legend  of 
Figure  1 .  Again,  CC  is  plotted  in  the  left  column.  MI  is  plotted  in  the  right  column.  From  the  top  to  bottom,  nearest  neighbor 
(NN),  tri-linear,  and  re-normalized  sine  interpolation  methods  are  used  to  obtain  floating  images,  respectively.  MI  curves  are 
much  peaked  than  CC,  especially  with  sine  and  tri-linear  interpolation.  The  voxel  is  isotropic  with  1.4  mm  on  a  side.  Image  data 
are  the  same  used  in  Figure  1 . 
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2.4.  Rigid  Body  Registration  Algorithm  with  Special  Features 

The  algorithm  includes  special  features  to  improve  robustness  for  registration  of  MR  prostate  images.1  Suppose  the 
iMRI  image  slice  is  the  reference  image,  the  matching  slice  extracted  from  the  high-resolution  MRI  volume  is  the 
floating  image.  We  use  a  multi-resolution  approach  and  perform  registration  from  low  to  high  resolution.  We  use  CC  at 
the  two  lower  resolutions  because  it  gives  fewer  local  maxima  and  because  it  can  be  calculated  faster  than  MI.  We  use 
MI  at  full  resolution  because  of  its  peaked  surface.  To  avoid  local  maxima,  we  include  a  restarting  feature  where 
registration  is  restarted  with  randomly  perturbed  parameters  obtained  from  a  uniform  distribution  about  the  initial 
transformation  values  at  the  current  resolution  being  used.  The  algorithm  restarts  until  the  absolute  CC  is  above  a 
threshold  of  0.5  as  experimentally  determined  or  the  maximum  number  of  restarts  is  reached.  Absolute  CC  is  used 
rather  than  MI  because  it  has  a  well-defined  range  between  0  and  1  and  because  it  provides  an  independent  check  of  the 
MI  result  at  the  highest  resolution.11 

We  record  all  important  results  following  an  optimization  cycle  including  the  CC  and/or  MI  values  and  the 
transformation  parameters.  At  the  end  of  processing  at  a  lower  resolution,  we  always  select  the  transformation 
parameters  having  the  maximum  CC  value.  We  then  scale  the  translation  parameters  appropriately  and  assign  the  new 
parameters  to  be  initial  values  at  the  next  higher  resolution.  At  the  highest  resolution,  MI  instead  of  CC  is  the  similarity 
measure,  and  we  select  the  final  transformation  parameters  to  be  those  having  the  maximum  MI  value. 

Additional  algorithm  details  are  now  described.  For  registration,  we  use  rigid  body  transformation  (three 
translations  and  three  rotations).  We  chose  tri-linear  interpolation  instead  of  sine  because  it  is  much  faster  and  because 
it  has  comparable  performance,  as  observed  from  Figures  1  and  2.  For  optimization,  we  use  the  downhill  simplex 
method  of  Nelder  and  Mead.17  Optimization  of  similarity  ends  either  when  the  maximum  number  of  calculations  is 
reached  (typically  500)  or  the  fractional  change  in  the  similarity  function  is  smaller  than  a  tolerance  (typically  0.001). 
The  input  MRI  volume  is  a  3D  MR  acquisition  giving  256  x  256  x  128  nearly  isotropic  voxels  over  a  field  of  view 
covering  the  whole  pelvis.  We  create  isotropic  voxels  of  about  1.4  mm  on  a  side  using  3D  linear  interpolation.  We  use 
IDL  (Interactive  Data  Language,  Research  System  Inc.,  Boulder,  CO.)  as  the  programming  language.  We  use  an  initial 
guess  assuming  an  identity  transformation,  i.e.,  all  initial  translation  and  rotation  parameters  are  zero,  because  the 
patient  is  normally  oriented  approximately  the  same  way  from  one  scan  to  the  next.  We  set  the  maximum  numbers  of 
restarts  at  10,  5,  and  3,  from  low  to  high  resolution,  respectively. 

2.5.  Simulation  of  iMRI  Slice  Images 

We  used  high-resolution  MRI  volumes  to  simulate  iMRI  images  by  creating  thick  slices  and  adding  noise  and  receive 
coil  inhomogeneity.11  Clinically,  we  typically  use  an  iMRI  slice  thickness  of  4.0  -  6.0  mm.  We  averaged  3  slices  1.4 
mm  thick  to  create  a  4.2  mm  thick  slice. 

We  added  noise  to  the  simulated  iMRI  image.  MR  noise  is  described  by  the  Rician  distribution,18  but  at  reasonably 
high  signal  values,  the  noise  is  accurately  approximated  with  Gaussian  white  noise.19  We  measured  typical  SNR  on  our 

*  9  ^  AA  A4 

open  magnet  system  using  a  homogenous  phantom  and  methods  described  elsewhere.  We  then  added  Gaussian 
noise  to  the  simulated  iMRI  slice  images  either  to  match  the  measured  SNR  or  to  give  much  greater  noise  to  further 
stress  registration.  We  report  noise  experiments  using  the  SNR  of  the  simulated  slice  images. 
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2.6.  Registration  Evaluation 

Our  standard  evaluation  method  was  to  compare  SV  and  W  registration.  Since  this  relies  on  W  registration  accuracy, 
we  now  review  our  previous  results.1  For  volume  pairs  acquired  over  a  short  time  span  from  a  supine  subject  with  legs 
flat  on  the  table,  prostates  were  well  aligned  and  prostate  centroid  displacements  were  typically  <  1  mm.  The  registra¬ 
tion  accuracy  as  determined  from  displacements  of  pelvic  bony  landmarks  was  1.6  ±0.2  mm.  This  error  might  be 
overestimated  because  it  includes  the  uncertainty  of  locating  the  bony  landmarks.  The  centroid  error  was  slightly 
smaller  because  the  prostate  was  at  the  volume  center  and  rotation  errors  had  less  effect.  From  our  success  with  W 
prostate  registration,  we  decided  that  we  could  obtain  SV  accuracy  by  comparing  to  W  registrations  for  those  volume 
pairs  having  low  VV  registration  error. 

To  compare  SV  and  W  registration,  we  defined  a  rectangular  volume  of  interest  (VOI)  just  covering  the  prostate 
and  calculated  voxel  displacements  between  the  two  registrations.  To  voxels  within  the  VOI,  we  applied  the 
transformations  obtained  by  W  and  by  SV  registrations.  We  then  calculated  the  3D  displacements  between  the 
transformed  voxels.  The  mean  voxel  distance  was  used  as  our  metric  of  SV  registration  error.  For  the  evaluation  of 
algorithm  robustness,  we  defined  the  SV  registration  as  being  successjul  when  the  3D  displacement  was  less  than  2.0 
mm. 


We  used  the  multiple  visualization  features  of  RegViz,  a  program  created  in  IDL  in  our  laboratory,  to  visually 
evaluate  registration  results.  First,  we  manually  segmented  prostate  boundaries  in  image  slices  and  copied  them  to 
corresponding  slices  from  the  other  volume.  This  enabled  visual  determination  of  the  overlap  of  prostate  boundaries 
over  the  entire  volume.  We  applied  the  same  method  to  evaluate  pelvic  registration.  Second,  color  overlay  displays 
were  used  to  evaluate  overlap  of  structures.  One  image  was  rendered  in  gray  and  the  other  in  the  “hot  -  iron”  color 
scheme  available  in  IDL.  To  visualize  potential  differences,  it  was  quite  useful  to  interactively  change  the  contribution 
of  each  image  using  the  transparency  scale.  Third,  we  used  a  sector  display,  which  divided  the  reference  and  registered 
images  into  rectangular  sectors  and  created  an  output  image  by  alternating  sectors  from  the  two  input  images.  Even 
subtle  shifts  of  edges  could  be  clearly  seen.1 


3.  EXPERIMENTAL  METHODS 


3.1.  Image  Acquisition 

All  MRI  volumes  were  acquired  using  a  1 .5  T  Siemens  MRI  system  (Magnetom  Symphony,  Siemens  Medical  Systems, 
Erlangen,  Germany).  An  8-element  phased  array  body  coil  was  used  to  ensure  coverage  of  the  prostate  with  a  uniform 
sensitivity.  Typically  two  anterior  and  two  posterior  elements  were  enabled  for  signal  acquisition.  We  used  two 
different  MR  sequences.  First,  we  used  a  3D  RF  spoiled  gradient  echo  steady  state  pulse  sequence  (FLASH)  with 
TR/TE/flip  parameters  of  12/5.0/60  which  give  256  x  256  x  128  voxels  over  a  330  x  330  x  256-mm  field  of  view  (FOV) 
to  yield  1.3  x  1.3  x  2.0-mm  voxels  oriented  to  give  the  highest  resolution  for  transverse  slices.  The  acquisition  time  was 
5.6  min.  This  sequence  was  good  for  pelvic  imaging  but  was  not  ideal  for  the  prostate.  Second,  we  used  a  3D  rapid 
gradient  echo  sequence  (PSIF)  designed  to  acquire  the  spin-echo  component  of  the  steady  state  response,  rather  then  the 
free  induction  decay.  The  spin  echo  component  formed  immediately  prior  to  the  RF  pulse,  and  it  was  shifted  toward  the 
prior  RF  pulse  through  appropriate  gradient  waveform  design.  The  sequence  with  9.4/5.0/60  (TR/TE/flip)  yielded 
160  x  256  x  128  voxels  over  a  219  x  350  x  192-mm  rectangular  FOV  and  1.4  x  1 .4  x  1.5-mm  voxels  oriented  to  give  the 
highest  resolution  for  transverse  slices.  There  was  over  sampling  at  31%  in  the  slice  direction  to  reduce  aliasing 
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artifacts.  The  acquisition  time  was  4.3  min.  Most  often,  we  used  the  second  sequence,  which  gave  excellent  image 
contrast  for  the  prostate  and  its  surroundings. 


3.2.  Imaging  Experiments 

We  acquired  high-resolution  MRI  volumes  from  three  volunteers.  For  each  volunteer,  three  image  volumes  were 
obtained  with  an  imaging  session.  Each  volume  was  acquired  with  compatible  conditions.  Volunteers  laid  supine  with 
legs  flat  similar  to  the  position  in  routine  MR  scanning.  Between  volume  acquisitions,  volunteers  got  off  the  MRI  table, 
stretched,  and  walked  around  to  ensure  that  they  would  assume  a  different  position  when  they  laid  back  on  the  table. 
The  coil  array  was  centered  on  the  prostate.  All  images  of  a  volunteer  were  acquired  with  the  same  MRI  acquisition 
parameters.  In  total,  there  are  9  pairs  of  high-resolution  MRI  volumes  for  registration. 


4.  RESULTS  AND  DISCUSSION 


4.1.  Slice  Orientation  and  Position 

As  described  in  Section  2,  we  obtained  relatively  low  noise,  high-resolution  MR  images  and  simulated  SV  registration 
results.  These  data  sets  allowed  us  to  test  effects  of  noise  and  receive  coil  inhomogeneity  in  a  controlled  fashion.  And, 
because  we  had  substantial  previous  experience  showing  the  accuracy  of  VV  registration  under  comparable  conditions, 
we  could  easily  determine  SV  error  by  comparing  results  to  W  registration. 

We  determined  SV  registration  results  for  slices  near  the  prostate  in  the  three  standard  orthogonal  orientations. 
Comparing  to  W,  mean  and  standard  deviation  registration  errors  across  9  volume  pairs  and  45  SV  registration 
experiments  were  0.4  mm  ±  0.2  mm,  0.5  mm  ±  0.2  mm,  and  2.6  mm  ±1.6  mm  for  transverse,  coronal  and  sagittal  slices 
covering  the  prostate,  respectively.  Transverse  slices  worked  best  because  they  contain  many  relatively  rigid  anatomical 
structures.  Coronal  slices  worked  next  best.  Sagittal  slices  gave  the  largest  error  because  they  contained  a  large  portion 
of  the  deformable  bladder  and  rectum.  We  further  found  that  transverse  slices  centered  on  the  prostate  produced  better 
results  than  those  above  or  below  the  prostate.  Image  slices  above  included  the  deformable  bladder  that  could  give  an 
inconsistent  structure  from  one  volume  to  the  next.  Image  slices  below  the  prostate  mainly  contained  muscle  and  fatty 
regions  from  the  hips  that  could  deform,  again  giving  inconsistent  image  data. 

4.2.  Noise  Level 

The  typical  iMRI  SNR  under  clinical  conditions  is  about  25.  Even  when  noise  much  exceeded  this  normal  situation, 
registration  results  were  quite  good.  A  100%  success  rate  was  achieved  with  an  acceptance  criterion  of  <  2.0  mm  even 
when  SNR  was  as  bad  as  10. 

4.3.  Robustness 

The  registration  algorithm  was  quite  robust  for  transverse  slices  covering  the  prostate.  Using  the  nine  volume  pairs  from 
three  volunteers,  the  algorithm  never  failed  for  any  transverse  slice  covering  the  prostate.  In  addition,  the  final 
registration  result  was  insensitive  to  initial  guesses  within  a  large  range,  [-60,  +60]  mm  for  translations  and  [-20,  +20] 
degrees  for  rotations.  With  the  restarting  algorithm,  we  even  successfully  registered  slices  as  much  as  80  mm  from  the 
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optimum.  This  working  range  is  probably  sufficient  for  clinical  applications  where  we  can  ensure  good  starting  values. 
Using  the  pelvic  bones  as  markers  and  needle  localization  methods,22  we  should  be  able  to  position  the  prostate  within 
about  ±  20  mm.  In  addition,  the  patient  normally  lies  supine  in  the  MR  bed  with  very  little  rotation  (<  ±  5  degrees). 

4.4.  Comparison  of  Different  Interpolation  Methods 

Computation  time  and  registration  accuracy  are  two  main  factors  to  consider  when  choosing  interpolation  methods. 
Using  tri-linear  interpolation,  the  time  for  an  SV  registration  was  typically  about  5  sec  on  a  Pentium  IV,  1.8  GHz  CPU, 
with  1Gbyte  of  memory.  If  the  re-normalized  sine  interpolation  method  was  used,  the  time  was  ~  10  min,  a  duration  not 
acceptable  for  our  application.  The  algorithm  was  written  in  IDL  and  could  probably  be  made  faster  in  a  lower  level 
language  such  as  C.  We  did  not  use  nearest  neighbor  because  of  the  obvious  insufficient  accuracy  as  induced  from  its 
flat  peak  of  the  similarity  curves  in  Figure  2.  A  call  to  the  Simplex  optimization  typically  resulted  in  50  to  105 
similarity  evaluations  before  the  tolerance  value  (0.001)  was  reached. 

4.5.  Comparison  of  Three  Alignment  Techniques 

When  registering  iMRI  image  slices  with  high-resolution  MR  volumes,  we  compared  three  alignment  techniques: 
(1)  an  iMRI  thick  slice  to  an  MR  thin  slice,  (2)  an  iMRI  thick  slice  to  the  average  of  three  MR  thin  slices,  and  (3)  three 
copies  of  the  iMRI  thick  slice  to  three  MR  thin  slices.  The  second  and  third  methods  were  tested  because  they  were 
thought  to  better  match  the  resolution  of  the  high  resolution  volume  to  the  the  thick  slide.  There  were  no  significant 
differences  (voxel  displacements  <  0.1  mm)  among  the  three  methods.  Hence,  we  used  the  first  method  because  of  its 
simplicity. 


5.  CONCLUSION 

The  slice  to  volume  registration  algorithm  is  quite  robust  for  transverse  slice  images  covering  the  prostate.  Since  live¬ 
time  iMRI  images  are  used  for  guidance  and  registered  images  are  used  for  adjunctive  information,  the  registration 
accuracy  is  very  probably  adequate.  It  is  quite  feasible  to  include  previously  acquired  high-resolution  MRI  or  nuclear 
images  into  iMRI-guided  treatment  procedures.  We  are  beginning  to  explore  this  application  in  animal  experiments. 
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Abstract  -  We  are  investigating  automatic  image  registration 
methods  that  can  be  used  for  interventional  magnetic  resonance 
imaging  (iMRI)  guided  radiofrequency  (RF)  thermal  ablation 
of  prostate  cancer.  We  tested  the  ability  of  slice-to-volume 
registration  between  iMRI  slice  images  and  high-resolution 
MRI  volumes.  Images  were  acquired  from  a  conventional  1.5  T 
and  an  interventional  0.2  T  MRI  system.  We  evaluated  the 
registration  quality  by  calculating  3D  displacement  on  a  voxel- 
by- voxel  basis  over  a  volume  of  interest  between  slice-to-volume 
registration  and  volume-to-volume  registration  that  was 
previously  shown  to  be  quite  accurate.  Visual  inspections  such 
as  color  overlay  and  contour  overlap  were  also  used  for 
registration  evaluation.  More  than  300  registration  experiments 
were  performed  on  MR  images  of  volunteers.  Results  showed 
that  the  registration  was  quite  robust  and  accurate  (<  2  mm)  for 
the  transverse  images  covering  the  prostate. 

Keywords:  medical  imaging,  image  registration,  interventional 
magnetic  resonance  imaging  (iMRI),  prostate  cancer. 

I.  Introduction 

We  currently  use  interventional  MRI  as  guidance  for  RF 
thermal  ablation  of  abdominal  cancer.  In  the  case  of  the 
prostate,  the  tumor  is  not  reliably  identified  with  MRI.  Other 
functional  imaging  techniques  are  required.  Potential 
methods  include  SPECT  imaging  and  MR  spectroscopy.  To 
incorporate  the  functional  images  with  interventional  MRI 
tumor  targeting,  one  can  first  register  the  low-resolution 
functional  images  with  high-resolution  MR  volumes.  Then 
by  registering  high-resolution  MR  volume  with  nearly  real¬ 
time  iMRI  acquisitions,  we  can  map  both  the  functional  data 
and  high-resolution  anatomic  information  to  interventional 
images  for  improved  tumor  targeting.  From  our  previous 
success  with  MR  volume-to-volume  (W)  registration  [1] 
and  slice-to-volume  (SV)  registration  using  simulated  iMRI 
images  [2],  we  are  further  testing  the  SV  registration  ability 
using  real  iMRI  images  in  this  study. 

n.  Materials  and  methods 

We  acquired  high-resolution  MR  volumes  from  three 
volunteers  using  a  3D  rapid  gradient  echo  sequence  (PSIF) 
with  9.4/5.0/60  (TR/TE/FA)  and  a  conventional  1.5  T  MR 
system  (Siemens  Magnetom  Symphony,Erlangen,  Germany). 
Volunteers  laid  supine  similar  to  the  diagnostic  position  in 
routine  MR  scanning.  Two  volumes  were  acquired  for  each 
volunteer.  Volume  images  were  normally  256x256x128  with 
the  voxel  size  of  1.4xl.4xl.5-mm. 

We  also  acquired  images  from  the  same  volunteers  using  a 
clinical  0.2  T  C-arm  interventional  MR  system  (Siemens 
Open  Symphony,  Erlangen,  Germany).  Volunteers  were 
supine  with  the  legs  supported  similar  to  the  lithotomy 
position  used  in  prostate  therapies.  This  position  should 
provide  access  for  needle  insertion  in  brachytherapy  or  RF 
thermal  ablation.  We  used  a  3D  PSIF  with  25/13/60 
(TR/TE/FA)  for  volume  acquisitions  and  2D  PSIF  with 
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15.2/7.4/45  (TR/TE/FA)  for  slice  acquisitions.  Slice  images 
were  128x128  with  pixel  size  of  2.8x2.8-mm  and  with 
effective  slice  thickness  of  5  mm.  For  each  volunteer,  we 
acquired  2  volumes  and  25  slices  covering  the  prostate.  They 
included  15  transverse,  5  coronal,  and  5  sagittal  slices.  We 
called  these  images  as  iMRI  volume  or  real  iMRI  slices.  We 
call  them  real  iMRI  slices  to  differentiate  from  previous 
experiments  on  simulated  iMRI  slices. 

We  performed  registration  experiments  using  an  automatic 
algorithm  with  special  features  important  for  robustness  for 
MR  pelvic  images  [1].  First,  we  performed  W  registration 
between  high-resolution  MRI  and  iMRI  volumes.  Second, 
we  extracted  slices  from  an  iMRI  volume  and  registered  the 
slices  with  the  high-resolution  MRI  volume.  We  evaluated 
SV  registrations  by  calculating  3D  displacement  on  a  voxel- 
by-voxel  basis  over  a  volume  of  interest  (VOI)  between  SV 
and  W  registration  [2].  Third,  we  tested  the  SV  registration 
between  real  iMRI  slices  and  high-resolution  MRI  volumes. 
We  used  visual  inspections  such  as  color  overlay  and  contour 
overlap  to  evaluate  the  SV  registration  quality. 

m.  Results 

We  performed  over  300  registration  experiments.  The  W 
registration  of  MRI  and  iMRI  volumes  is  quite  accurate  with 
prostate  centroid  typically  less  than  1  mm.  As  compared  to 
the  W  results,  the  VOI  displacement  of  SV  registration  is 
less  than  2  mm  for  all  transverse  slices  covering  the  prostate. 
Most  coronal  slices  (86%)  gave  an  accuracy  of  less  than 
2.5  mm.  Nearly  half  of  sagittal  slices  (46%)  gave  an 
accuracy  of  less  than  5  mm.  This  suggested  that  transverse 
slices  were  the  best.  Inspection  indicates  that  bladder  and/or 
rectal  filling  interfered  with  registration  of  coronal  and 
sagittal  slices  because  of  deformation.  Visual  inspection  of 
SV  registration  between  iMRI  slices  and  high-resolution 
MRI  volumes  demonstrated  similar  results. 

IV.  Conclusion 

It  is  quite  feasible  to  use  SV  image  registration  to  aid  the 
application  of  interventional  MRI-guided  RF  thermal 
ablation  of  prostate  cancer.  We  are  beginning  to  explore 
these  applications  in  dog  experiments. 
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ABSTRACT 

We  are  investigating  interventional  MRI  guided  radiofrequency  (RF)  thermal  ablation  for  the  minimally  invasive 
treatment  of  prostate  cancer.  Among  many  potential  applications  of  registration,  we  wish  to  compare  registered  MR 
images  acquired  before  and  immediately  after  RF  ablation  in  order  to  determine  whether  a  tumor  is  adequately  treated. 
Warping  registration  is  desired  to  correct  for  potential  deformations  of  the  pelvic  region  and  movement  of  the  prostate. 
We  created  a  two-step,  three-dimensional  (3D)  registration  algorithm  using  mutual  information  and  thin  plate  spline 
(TPS)  warping  for  MR  images.  First,  automatic  rigid  body  registration  was  used  to  capture  the  global  transformation. 
Second,  local  warping  registration  was  applied.  Interactively  placed  control  points  were  automatically  optimized  by 
maximizing  the  mutual  information  of  corresponding  voxels  in  small  volumes  of  interest  and  by  using  a  3D  TPS  to 
express  the  deformation  throughout  the  image  volume.  Images  were  acquired  from  healthy  volunteers  in  different 
conditions  simulating  potential  applications.  A  variety  of  evaluation  methods  showed  that  warping  consistently  improved 
registration  for  volume  pairs  whenever  patient  position  or  condition  was  purposely  changed  between  acquisitions.  A  TPS 
transformation  based  on  180  control  points  generated  excellent  warping  throughout  the  pelvis  following  rigid  body 
registration.  The  prostate  centroid  displacement  for  a  typical  volume  pair  was  reduced  from  3.4  mm  to  0.6  mm  when 
warping  was  added. 

Keywords:  Warping  image  registration,  medical  imaging,  interventional  magnetic  resonance  imaging  (iMRI),  mutual 
information,  thin  plate  spline,  prostate  cancer. 


1.  INTRODUCTION 

We  are  investigating  three-dimensional  (3D)  image  registration  to  be  used  in  applications  of  prostate  cancer  diagnosis, 
staging,  and  therapy.  In  particular,  we  are  interested  in  applications  related  to  the  minimally  invasive  interventional 
magnetic  resonance  imaging  (iMRI)  guided  treatment  of  prostate  cancer.  At  our  institution,  we  currently  use  iMRI  on  a 
low-field  open  magnet  system  to  guide  radiofrequency  (RF)  thermal  ablation  of  abdominal  cancer,1,2  and  we  are 
investigating  this  method  for  prostate  cancer  treatment.  A  unique  feature  of  iMRI -guided  thermal  ablation  is  that  therapy 
can  be  monitored  with  MR  either  by  acquiring  images  of  the  thermally  induced  lesion  or  by  measuring  temperature.  In 
addition,  MR  imaging  of  the  prostate  is  desirable  because  it  more  accurately  delineates  the  prostate  than  does  CT,3,4 
which  can  overestimate  the  prostate  volume,5,6  and  ultrasound,  which  has  a  tendency  to  underestimate  the  extent  of 
lesions.7 

Several  applications  in  prostate  imaging  require  registration.  First,  comparison  of  registered  MR  images  acquired 
before  and  immediately  after  RF  ablation  can  be  used  to  determine  whether  a  tumor  is  adequately  treated.  This  is 
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particularly  helpful  in  instances  where  the  edematous  response  to  treatment  can  be  confused  with  a  highly  perfused 
tumor.  Second,  registration  of  serial  examinations  can  be  used  to  follow  regression/progression  of  tumor.  Third, 
incorporating  the  functional,  biochemical  images  into  the  iMRI  paradigm  will  aid  image-guided  treatments,  as  recently 
reported  by  us.8,9 

Several  reports  describe  methods  for  registration  in  the  pelvis  or  prostate. !(M5  These  methods  required  either 
segmentation  or  visual  identification  of  structures  and  they  were  based  on  rigid  body  transformation.  We  recently 
reported  a  mutual  information  rigid  body  transformation  method  for  prostate  registration.16  For  volume  pairs  acquired 
over  a  short  time  span  from  a  supine  subject  with  legs  flat  on  the  table,  registration  accuracy  of  both  prostate  centroids 
(typically  <  1  mm)  and  bony  landmarks  (average  1.6  mm)  was  on  the  order  of  a  voxel  (~  1.4  mm).  We  obtained 
somewhat  larger  prostate  registration  errors  of  about  3.0  mm  when  volume  pairs  were  obtained  under  very  different 
conditions,  e.g.,  legs  flat  and  legs  raised,  or  with  and  without  bladder  or  rectal  filling.  Rigid  body  registration  of  the 
pelvis  cannot  follow  prostate  movements  due  to  changes  in  the  postures  of  legs  and  deformation  of  the  bladder  and 
rectum,  as  reported  by  us  16  and  others.14,17 

There  are  a  number  of  reports  on  warping  registration.18'20  These  applications  were  mainly  on  the  brain  and  breast; 
and  far  few  described  results  for  the  abdomen.21’22  For  our  application,  we  are  developing  voxel-based  warping 
registration  in  the  pelvis  and  prostate. 

There  are  challenges  to  the  pelvis  and  prostate  registration.  First,  pelvic  regions  can  change  shape  significantly, 
unlike  the  brain  to  which  registration  has  been  most  often  applied.  Different  patient  positions  such  as  displacement  of  the 
legs  can  cause  the  movement  and  deformation  of  internal  organs.  Second,  the  normal  prostate  is  a  small  organ  that  when 
healthy  measures  only  about  3.8  cm  in  its  widest  dimension  transversely  across  the  base  23  Third,  the  prostate  might 
move  relative  to  the  pelvic  bones  due  to  changes  in  bladder  and  rectal  filling.14,17 

In  the  present  study,  we  perform  experiments  to  determine  if  warping  can  improve  registration  of  pelvic  MR.  High 
quality,  3D  MR  image  volumes  from  a  commercially  available  1 .5  T  system  are  used  to  determine  the  best  possible 
results.  We  examine  conditions  found  in  the  potential  clinical  applications  described  previously.  We  will  qualitatively 
and  quantitatively  compare  results  of  warping  and  rigid  body  registration  using  9  volume  pairs  from  three  volunteers. 


2.  REGISTRATION  ALGORITHM 


2.1.  Similarity  Measurements 

We  used  two  similarity  measures,  mutual  information  and  correlation  coefficient  (CC),  in  our  registration.  One  volume  R 
is  the  reference ,  and  the  other  F  \s  floating.  Their  mutual  information  MI(R,  F )  is  given  below.24 

MI(R,F)  =  Xp„(r,/)log  ^(r,/)- 
tf  Pr(t)pf(J ) 

The  joint  probability  and  the  marginal  probabilities  pR (r)  of  the  reference  image  and  pF(f) of  the 

floating  image,  can  be  estimated  from  the  normalized  joint  intensity  histograms.  When  two  images  are  geometrically 
aligned,  MI  is  maximal.24  The  correlation  coefficient  CC(R,F)  is  given  below.25 

cm. f)=  ,  wc /)-?(/» 
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Here  /?(r),  F (j")  denote  the  average  intensities  of  the  reference  and  floating  volumes  and  the  summation  includes  all 
voxels  within  the  overlap  of  both  volumes. 


2.2.  Rigid  Body  Registration  Algorithm  with  Special  Features 

The  rigid  body  registration  algorithm  includes  special  features  to  improve  robustness  for  MR  pelvic  images.  Reference  is 
matched  to  the  reformatted  volume  obtained  by  transforming  the  floating  volume.  We  use  a  multi-resolution  approach 
and  perform  registration  from  low  to  high  resolution.  At  low  resolution,  we  resample  both  images  at  1/4  or  1/2  number  of 
voxels  along  each  linear  dimension,  respectively.  A  simplex  algorithm  varies  the  six  rigid  body  transformation 
parameters  (three  translations  and  three  angles)  to  optimize  the  similarity  measure.26  We  use  CC  at  the  two  lower 
resolutions  because  it  gives  fewer  local  maximums  8,9,16  and  because  it  can  be  calculated  faster  than  MI.  We  use  MI  at 
lull  resolution  because  the  peaked  similarity  function  gives  a  more  precise  solution  than  CC.9,16  To  avoid  local 
maximums,  we  include  a  restarting  feature  where  registration  is  restarted  with  randomly  perturbed  parameters.  The 
algorithm  restarts  until  the  absolute  CC  is  above  an  experimentally  determined  threshold  or  the  maximum  number  of 
restarts  is  reached.  Absolute  CC  is  used  for  the  restart  test  rather  than  MI  because  CC  has  fewer  problems  with  local  and 
incorrect  global  maximums  at  low  resolution.9,16 

We  record  all  important  results  following  an  optimization  cycle  including  the  CC  and/or  MI  values,  the  number  of 
restarts,  and  the  transformation  parameters.  At  the  end  of  processing  at  a  lower  resolution,  we  always  select  the 
transform-ation  parameters  having  the  maximum  CC  value.  We  then  scale  the  translation  parameters  appropriately  and 
assign  the  new  parameters  to  be  initial  values  at  the  next  higher  resolution.  At  die  highest  resolution,  we  select  the  final 
transformation  parameters  to  be  those  with  the  maximum  MI  value. 

Typical  parameter  values  are  now  described.  We  use  an  initial  guess  at  the  lowest  resolution  of  all  zeros  because  the 
patient  is  normally  oriented  approximately  the  same  way  from  one  scan  to  the  next.  Based  on  experience,  we  set  the  CC 
thresholds  at  0.50,  0.55,  and  0.65,  and  the  maximum  numbers  of  restarts  at  10,  5,  and  3,  from  low  to  high  resolution, 
respectively. 


2.3.  Warping  Registration  Using  Optimized  Control  Points 

The  warping  registration  algorithm  includes  three  major  steps:  control  point  (CP)  selection,  control  point  optimization, 
and  thin  plate  spline  (TPS)  warping.  Again,  the  unchanging  volume  is  the  reference,  and  the  one  to  be  warped  \s  floating. 

The  manual  selection  of  CP’s  is  an  important  step.  We  used  RegViz,  a  program  written  in  IDL  (Interactive  Data 
Language,  Research  System  Inc.,  Boulder,  CO)  and  created  in  our  laboratory  for  visualizing  and  analyzing  image 
volumes.  Following  rigid  body  registration,  the  aligned  two  volumes  are  displayed  in  two  rows  slice-by-slice. 
Corresponding  CP's  in  the  two  volumes  are  placed  using  a  cursor,  and  sometimes  they  are  in  different  image  slices.  The 
3D  coordinates  are  automatically  stored  in  a  file.  Because  of  the  optimization  that  occurs  later,  the  correspondence  can  be 
up  to  15  mm  or  =  10  voxels  in  error.  More  experience  with  CP  selection  is  described  in  Results.  Typically,  we  used  180 
CP's  for  a  volume  with  256  x  256  x  140  isotropic  voxels. 

The  next  step  of  the  warping  algorithm  is  the  CP  optimization.  We  define  a  small  cubic  volume  of  interest  (VOI) 
centered  at  each  CP.  The  VOI  can  be  16  or  64  voxels  on  a  side.  A  simplex  optimization  algorithm  varies  the  x,  y,  and  z 
transformation  parameters  of  the  floating  VOI  until  the  mutual  information  with  the  reference  VOI  is  optimized.  Each 
control  point  is  optimized  independently. 
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The  final  major  step  is  to  warp  the  floating  volume  using  the  optimal  CP's  coordinates  to  establish  a  TPS 
transformation.  We  extended  the  two-dimensional  TPS  transformation  method  reported  by  Bookstein27  to  three 
dimensions  in  a  manner  similar  to  that  of  Davis  et  al.19 


2.4.  Additional  Details 

There  are  several  preprocessing  details.  The  input  MRI  volume  is  a  3D  MR  acquisition  giving  256  x  256  x  128  nearly 
isotropic  voxels  over  a  field  of  view  covering  the  whole  pelvis.  We  create  isotropic  voxels  of  about  1.4  mm  on  a  side 
using  3D  linear  interpolation.  From  the  top  and  bottom  of  the  volume,  we  optionally  crop  transverse  slices  that  are  over 
35  mm  away  from  the  prostate  rim.  Cropping  is  done  to  remove  slices  having  reduced  brightness  due  to  sensitivity  fall 
off  from  the  receiver  coils  and/or  artifacts  from  a  small  field  of  view.  For  both  rigid  body  registration  and  VOI 
optimization,  we  use  trilinear  interpolation.  Optimization  of  similarity  ends  either  when  the  maximum  number  of 
calculations  is  reached  (typically  500)  or  the  fractional  change  in  the  similarity  function  is  smaller  than  a  tolerance 
(typically  0.001).  We  use  IDL  as  the  programming  language. 


3.  EXPERIMENTAL  METHODS 


3.1.  Image  Acquisition 

All  MRI  volumes  were  acquired  using  a  1.5  T  Siemens  MRI  system  (Magnetom  Symphony,  Siemens  Medical  Systems, 
Erlangen,  Germany).  An  8-element  phased  array  body  coil  was  used  to  ensure  coverage  of  the  prostate  with  a  uniform 
sensitivity.  Typically  two  anterior  and  two  posterior  elements  were  enabled  for  signal  acquisition.  We  used  two  different 
MR  sequences.  First,  we  used  a  3D  RF  spoiled  gradient  echo  steady  state  pulse  sequence  (FLASH)  with  TR/TE/flip 
parameters  of  12/5.0/60  which  give  256  x  256  x  128  voxels  over  a  330  x  330  x  256-mm  field  of  view  (FOV)  to  yield 
1 .3  x  1 .3  x  2.0-mm  voxels  oriented  to  give  the  highest  resolution  for  transverse  slices.  The  acquisition  time  was  5.6  min 
This  sequence  was  good  for  pelvic  imaging  but  was  not  ideal  for  the  prostate.  Second,  we  used  a  3D  rapid  gradient  echo 
sequence  (PSIF)  designed  to  acquire  the  spin-echo  component  of  the  steady  state  response,  rather  then  the  free  induction 
decay.  The  spin  echo  component  formed  immediately  prior  to  the  RF  pulse,  and  it  was  shifted  toward  the  prior  RF  pulse 
through  appropriate  gradient  waveform  design.  The  sequence  with  9.4/5.0/60  (TR/TE/flip)  yielded  160  x  256  x  128 
voxels  over  a  219  x  350  x  192-mm  rectangular  FOV  and  1.4  x  1.4  x  1.5-mm  voxels  oriented  to  give  the  highest 
resolution  for  transverse  slices.  There  was  over  sampling  at  31%  in  the  slice  direction  to  reduce  aliasing  artifacts.  The 
acquisition  time  was  4.3  min.  Most  often,  we  used  the  second  sequence,  which  gave  excellent  image  contrast  for  the 
prostate  and  its  surroundings. 


3.2.  Imaging  Experiments 

We  acquired  3D  MRI  volume  images  from  three  normal  volunteers  under  a  variety  of  conditions  simulating  anticipated 
conditions  in  diagnostic  and  treatment  applications.  Before  image  acquisition,  each  volunteer  drank  water  and  had  a 
relatively  full  bladder.  In  the  diagnostic  position,  the  subject  laid  supine  throughout  MR  scanning.  In  the  treatment 
position,  the  subject  was  supine,  and  his  legs  were  supported  at  30°-60°  relative  to  the  horizon  and  separated  in  a  “V” 
with  an  angle  of  60°-90°  between  two  legs.  This  position  should  provide  access  for  needle  insertion  in  brachytherapy  or 
RF  thermal  ablation.  In  some  experiments,  the  subject  micturated  to  create  an  empty  bladder  prior  to  imaging.  For  each 
subject,  image  volumes  were  obtained  on  the  same  day  within  one  imaging  session.  Between  volume  acquisitions, 
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volunteers  got  off  the  MRI  table,  stretched,  and  walked  around  to  ensure  that  they  would  assume  a  different  position 
when  they  laid  back  on  the  table.  All  images  of  a  volunteer  were  acquired  with  the  same  MRI  acquisition  parameters  so 
as  to  ensure  very  similar  gray  values.  In  total,  there  are  3  volumes  for  each  volunteer. 


3*3.  Volume  Pairs  for  Registration 

We  registered  9  volume  pairs  under  different  conditions.  Three  pairs  are  treatment-diagnosis;  three  pairs  are  full-empty 
bladder;  and  three  pairs  are  diagnosis-diagnosis.  Rigid  body  and  warping  registration  were  applied  to  each  of  the  volume 
pairs.  Results  were  evaluated  as  described  next. 


3.4.  Registration  Evaluation 

We  used  the  multiple  visualization  features  of  RegViz  to  visually  evaluate  registration  results.  First,  we  manually 
segmented  prostate  boundaries  in  image  slices  and  copied  them  to  corresponding  slices  from  the  other  volume.  This 
enabled  visual  determination  of  the  overlap  of  prostate  boundaries  over  the  entire  volume.  We  applied  the  same  method 
to  evaluate  pelvic  registration.  Second,  color  overlay  displays  were  used  to  evaluate  overlap  of  structures.  One  image  was 
rendered  in  gray  and  the  other  in  the  “hot  -  iron”  color  scheme  available  in  IDL.  To  visualize  potential  differences,  it  was 
quite  useful  to  interactively  change  the  contribution  of  each  image  using  the  transparency  scale.  Third,  we  used  a  sector 
display,  which  divided  the  reference  and  registered  images  into  rectangular  sectors  and  created  an  output  image  by 
alternating  sectors  from  the  two  input  images.  Even  subtle  shifts  of  edges  could  be  clearly  seen.16 

Voxel  gray  value  measures  were  calculated  as  indicators  of  regitration  quality.  Mutual  information  and  correlation 
coefficient  between  registered  volumes  were  computed.  The  higher  the  MI  values,  the  more  likely  the  two  volumes  are 
well  registered.  Since  volumes  to  be  registered  were  acquired  using  the  same  acquisition  parameters,  high  absolute  CC 
values  were  obtained  when  registration  was  good.20  Because  voxel  intensities  were  comparable,  we  created  difference 
images  and  calculated  statistics  such  as  the  voxel  mean  and  standard  deviation  following  registration. 

Finally,  to  assess  prostate  registration  error,  we  measured  potential  displacements  of  the  3D  centroid  of  the  prostate. 
We  used  RegViz  to  manually  segment  the  prostate  across  all  image  slices  and  calculated  the  3D  centroid,  A  similar 
method  was  used  by  West  et  al.  to  evaluate  rigid  body  registration  of  the  brain.28 


4.  RESULTS  AND  DISCUSSION 


4.1.  Control  Point  Selection 

In  well  over  50  registration  experiments,  we  investigated  the  number  and  placement  of  CP’s  on  warping  registration 
quality.  We  quickly  determined  that  many  CP’s  are  required  in  order  to  get  good  visual  matching  of  our  very  high- 
resolution  MR  images  showing  great  anatomical  detail,  especially  when  large  deformations  occurred  as  in  the  case  of 
diagnosis-treatment.  We  progressively  increased  the  number  of  CP’s  from  15  to  250.  We  found  that  less  than  120  CP’s 
did  not  produce  smooth  and  reasonable  warps.  Warping  with  180  CP's  excellently  approximated  the  deformation  of  the 
pelvis  and  internal  organs.  More  than  220  CP’s  did  not  give  significant  improvement  but  needed  more  time  for  manual 
selection  and  optimization.  As  a  result  of  this  experience,  we  modified  the  registration  method  to  be  more  suitable  for 
many  CP’s  (see  Section  2). 
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Figure  I.  Control  point  selection  when  images  are  acquired  in  the  treatment  and  diagnostic  positions.  Image  (a)  is  from 
the  reference  volume  acquired  in  the  treatment  position  with  legs  raised.  Image  (b)  is  to  be  warped  and  is  from  the  volume 
acquired  in  the  diagnostic  position  with  the  subject  supine  on  the  table.  Transverse  slices  best  show  the  deformations, 
especially  at  the  legs.  As  described  in  the  text,  control  points  indicated  by  the  white  dots  are  selected  around  the  pelvic 
surface  and  the  prostate.  Each  control  point  is  located  at  one  voxel  but  displayed  much  bigger  for  better  visualization. 
Volumes  are  from  volunteer  S2. 


We  also  determined  that  it  was  necessary  to  place  the  CP’s  in  strategic  locations,  and  some  rules  follow.  For 
registration  of  treatment  and  diagnostic  image  volumes,  most  CP’s  were  selected  using  transverse  slices  because  they 
best  showed  the  pelvic  displacement  when  moving  the  legs  to  the  treatment  position  (Figure  1).  About  30  CP  pairs  were 
placed  near  edge  and  point  features  having  recognizable  correspondence  on  each  of  5-8  transverse  slices  with  a  z  interval 
of“  8  mm,  covering  the  entire  prostate  region.  Additionally,  we  placed  about  25  CPs  from  sagittal  slices  because  they 
provided  other  structures  that  can  be  missed  in  the  transverse  images.  It  was  also  important  to  include  CP's  from  organs 
other  than  the  prostate  because  they  constrained  warps.  We  always  placed  CP's  at  critical  regions  such  as  the  prostate 
center,  pelvic  surface,  bladder  border,  and  rectal  walls. 


4.2.  Registration  Quality  of  Warping  and  Rigid  Body  Registration 

In  Figure  2,  we  compare  warping  and  rigid  body  registration  for  a  typical  volume  pair  in  the  treatment  and  diagnostic 
positions.  Using  rigid  body  registration,  there  is  significant  misalignment  throughout  large  regions  in  the  pelvis  (Figure 
2a)  that  is  greatly  reduced  with  warping  (Figure  2b).  Note  that  waiping  even  allows  the  outer  surfaces  to  match  well. 
Other  visualization  methods  such  as  two-color  overlays  and  difference  images,  quickly  show  matching  of  structures 
without  segmentation  but  do  not  reproduce  well  on  a  printed  page.  The  prostate  3D  centroid  calculated  from  segmented 
images  displaced  by  only  0.6  mm,  or  0.4  voxels,  following  warping.  Following  rigid  body  registration,  the  prostate  was 
misaligned  with  a  displacement  to  the  posterior  of  «  3.4  mm  when  in  the  treatment  position 

We  examined  the  effect  of  conditions  such  as  bladder  and  rectal  filling  that  might  change  from  one  imaging  session 
to  the  next  (not  shown).  Contour  overlap  showed  that  warping  registration  perfectly  aligns  the  prostate  while  rigid  body 
does  not.  In  addition,  rigid  body  registration  does  not  align  the  bladder.  With  warping,  the  bladder  perfectly  matches  the 
reference.  Other  visualization  methods  showed  excellent  alignment  of  internal  and  surface  edges.  Difference  images 
show  that  warping  greatly  improves  alignment  of  internal  structures  as  compared  to  rigid  body  registration. 

We  also  examined  volume  pairs  with  both  volumes  acquired  in  the  diagnostic  position  under  comparable  conditions. 
In  all  such  cases,  rigid  body  registration  worked  as  well  as  warping.  There  were  no  noticeable  deformations  in  the  pelvis, 
and  prostate  centroids  typically  displaced  less  than  1.0  mm  between  the  two  registered  volumes. 
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Figure  2.  Comparison  of  warping  and  rigid  body  registration  for  volumes  acquired  in  the  treatment  and  diagnostic 
positions.  Image  (a)  is  from  the  floating  volume  acquired  in  the  diagnostic  position  following  rigid  body  registration. 
Pelvic  contours  manually  segmented  from  the  reference  show  significant  misalignment.  Image  (b)  is  from  the  floating 
volume  acquired  in  the  diagnostic  position  following  warping  registration.  The  misalignment  is  greatly  improved  with 
warping.  Images  are  transverse  slices  from  subject  S2. 
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Figure  3.  Image  statistics  of  absolute  intensity  difference  images  for  rigid  body  and  warping  registration.  The  mean  (a)  and 
standard  deviation  (b)  are  plotted.  See  the  legend  of  Figure  3  for  other  details.  Waiping  decreased  the  mean  and  standard 
deviation  in  each  case,  but  the  most  significant  decreases  occurred  in  the  case  of  the  treatment-diagnosis  volume  pairs. 
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4.3.  Quantitative  Evaluation  of  Warping  Registration 

Statistics  of  image  differences  following  rigid  body  and  warping  registration  are  shown  in  Figure  3.  Warping  reduces  the 
absolute  intensity  difference  between  corresponding  voxels  (Figure  3a).  We  did  not  use  non-absolute  differences  as 
evaluation  criteria  because  the  positive  and  negative  differences  can  be  canceled.  The  standard  deviation  of  absolute 
difference  is  also  reduced  (Figure  3b).  We  also  calculated  the  correlation  coefficient  and  mutual  information  values 
between  registered  volumes.  Warping  increased  CC  and  MI  values  in  every  case. 


4.4.  Algorithmic  Robustness  and  Efficiency 

The  rigid  body  algorithm  gives  robust  registration.  Because  of  two  principal  design  features,  the  algorithm  is  quite  robust 
and  accurate  for  volume  pairs  acquired  in  the  same  positions  and  with  comparable  conditions.16  First,  using  both  CC  and 
MI  at  different  resolutions  was  an  important  feature  that  increased  robustness.  CC  gave  fewer  local  minimums  at  low 
resolutions  and  MI  was  more  accurate  at  high  resolution.8,9  Second,  the  restarting  mechanism  was  also  quite  important. 
Without  restarting,  we  found  that  registrations  sometimes  failed  in  cases  of  volumes  with  large  mismatches  and 
significant  deformation.  Even  these  cases  resulted  in  a  proper  solution  when  restarting  was  employed.  The  time  for  rigid 
body  registration,  typically  5  minutes  on  a  Pentium  IV,  1.8  GHz  CPU,  with  1.0  GBytes  of  memory,  could  be  greatly 
improved  with  C  rather  than  IDL. 

The  interactive  warping  registration  algorithm  is  designed  to  be  very  computationally  efficient  for  TPS  warping  with 
hundreds  of  CP's.  First,  the  optimization  of  small  VOI’s  is  very  fast.  Second,  we  optimized  each  CP  separately  because 
the  optimization  of  three  parameters  (x,  yf  and  z)  is  simple  and  fast.  Third,  we  applied  the  TPS  transformation  once  to  the 
final,  optimal  CP's;  this  saved  considerable  time.  Using  the  same  computer  above,  for  a  volume  with  256x256x140 
voxels  and  180  CP's,  the  warping  registration  typically  takes  about  15-45  minutes  depending  on  the  VOI  size.  If  we  were 
to  use  optimized  C  code,  the  total  time  for  rigid  body  and  warping  registration  should  reduce  to  within  5  minutes. 


5.  CONCLUSION 

The  mutual  information  warping  registration  works  better  than  rigid  body  registration  whenever  the  subject  position  or 
condition  is  greatly  changed  between  acquisitions.  It  will  probably  be  a  useful  tool  for  many  applications  in  prostate 
diagnosis,  staging,  and  therapy.  It  is  fast  for  hundreds  of  control  points  and  can  be  applied  to  a  variety  of  applications. 
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ABSTRACT 

We  are  investigating  methods  to  register  live-time  interventional  magnetic  resonance  imaging  (iMRI)  slice  images  with  a 
previously  obtained,  high  resolution  MRI  image  volume.  The  immediate  application  is  for  iMRI-guided  treatments  of 
prostate  cancer.  We  created  and  evaluated  a  slice-to-volume  mutual  information  registration  algorithm  for  MR  images  with 
special  features  to  improve  robustness.  Features  included  a  multi-resolution  approach  and  automatic  restarting  to  avoid  local 
minima.  We  acquired  3D  volume  images  from  a  1.5  T  MRI  system  and  simulated  iMRI  images.  To  assess  the  quality  of 
registration,  we  calculated  3D  displacement  on  a  voxel-by-voxel  basis  over  a  volume  of  interest  between  slice-to-volume 
registration  and  volume-to-volume  registrations  that  were  previously  shown  to  be  quite  accurate.  More  than  500  registration 
experiments  were  performed  on  MR  images  of  volunteers.  The  slice-to-volume  registration  algorithm  was  very  robust  for 
transverse  slice  images  covering  the  prostate.  A  100%  success  rate  was  achieved  with  an  acceptance  criterion  of  <  1.0  mm 
displacement  error  over  the  prostate.  Our  automatic  slice-to-volume  mutual  information  registration  algorithm  is  robust  and 
probably  sufficiently  accurate  to  aid  in  the  application  of  iMRI-guided  thermal  ablation  of  prostate  cancer. 

Keywords:  Image  registration,  mutual  information,  magnetic  resonance  imaging  (MRI),  interventional  MRI,  prostate 
cancer,  minimally  invasive  treatment,  thermal  ablation. 


1.  INTRODUCTION 

We  are  investigating  methods  for  registering  slice  images  obtained  on  a  low  field  iMRI  system  to  high-resolution  MR 
volumes  obtained  on  a  traditional  1.5  T  scanner.  Immediate  applications  involve  the  treatment  of  prostate  cancer  as 
described  below.  To  be  useful  for  an  interventional  procedure,  a  registration  method  must  be  automatic,  accurate,  robust,  and 
fast.  Currently,  we  are  investigating  voxel-based  methods  because  of  their  ease  of  use  and  reported  accuracy  for  other 
applications. 

There  are  important  reasons  for  this  investigation.  At  our  institution,  we  have  extensive  experience  with  minimally 
invasive  treatment  of  abdominal  cancer  using  iMRI-guided  radiofrequency  thermal  abaltion. 1,2,3  In  the  case  of  the  prostate, 
the  tumor  is  not  reliably  identified  with  MR.  Hence,  other  functional  imaging  techniques  are  required,  and  potential  methods 
include  SPECT  antibody  imaging  and  MR  spectroscopy.  To  incorporate  the  functional  images  with  iMRI  tumor  targeting, 
one  can  first  register  the  low-resolution  functional  images  with  a  high-resolution  MRI.  Then  by  registering  the  high- 
resolution  MR  volume  with  live-time  iMRI  acquisitions,  we  can  map  both  the  functional  data  and  high-resolution  anatomic 
information  to  live-time  iMRI  images  for  improved  tumor  targeting.  Image  guided  biopsies  are  another  important  application 
of  iMRI  in  those  cases  where  the  tumor  is  more  readily  seen  in  MR  than  in  CT  or  ultrasound.  4,5,6 

Previously  reported  methods  on  slice  to  volume  registration  were  mainly  applied  to  the  brain. 7,8,9  For  the  case  of  volume 
to  volume  registration,  there  are  many  reports  of  accurate  voxel-based  registration  in  the  brain  10,11,12,13  and  abdominal  and 
pelvic  organs. 1,1 4 
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There  are  challenges  to  successful  slice  to  volume  registration  for  the  prostate.  They  include: 

1)  A  single  slice,  or  a  few  slices,  provides  much  less  information  than  an  entire  volume  for  voxel  based  matching. 

2)  There  is  a  low  signal  to  noise  ratio  (SNR)  in  iMRI  because  of  the  emphasis  on  fast  imaging  typically  with  a  low  field 
scanner. 

3)  The  pelvic  region  has  irregular  boundaries  and  can  deform,  unlike  the  brain  to  which  registration  has  been  most  often 
applied.  The  prostate  might  move  relative  to  the  pelvic  bones  due  to  changes  in  rectal  and  bladder  filling  15,16  or 
movement  of  the  patient  for  treatment. 

4)  The  normal  prostate  is  a  small  organ  that  when  healthy  measures  only  about  3.8  cm  in  its  widest  dimension  transversely 
across  the  base, 17  and  the  small  prostate  is  located  below  a  much  larger  bladder  that  can  change  shape  and  size. 

5)  The  non-homogenous  receive  coil  response  can  change  from  one  acquisition  to  next. 

We  are  investigating  the  use  of  voxel-based  registration  for  this  important  application.  We  created  a  mutual  information 

algorithm  modified  to  include  some  features  to  improve  robustness.  We  performed  registration  experiments  under  conditions 

found  with  low  field  open  MR  imaging. 


2.  REGISTRATION  ALGORITHM 


2.1.  Similarity  Measurement 

Two  similarity  measurements,  mutual  information  (MI)  and  correlation  coefficient  (CC),  are  used  in  our  algorithm.  The 
Ml-based  registration  method  is  robust  and  suitable  for  multi-modality  registration,  is  highly  accurate  for  brain 
registration,18  and  is  suitable  for  abdominal  registration  where  there  can  be  deformation.1,14  However,  the  MI  method  has  the 
problem  of  interpolation  artifacts,  which  are  especially  serious  in  the  case  of  down  sampling  in  a  multi-resolution  approach.19 
Fortunately,  CC  produces  fewer  locally  optimum  values  than  does  MI.20  Our  method  combines  both  similarity  measures  to 
use  the  good  attributes  of  each. 

MI  quantifies  interdependency  of  two  variables,  such  as  image  gray  intensities  of  image  A  and  B.  When  A  and  B  are 
exactly  aligned,  MI  is  maximal.18  We  calculate  MI  using  Equations  9  to  12  in  the  report  by  Maes  et  al.18  CC  is  a  measure  of 
the  relation  between  the  statistical  distributions  of  the  two  images.  The  absolute  of  CC  has  the  range  of  0  to  1 .  The  higher  the 
absolute  value  of  CC,  the  more  dependence  the  two  images  have.21  We  calculate  CC  using  the  equation  10  in  the  report  by 
Rueckert  et  al.21 

The  algorithm  shown  in  Figure  1  included  special  features  to  improve  robustness  for  registration  of  MR  prostate  images. 
In  the  pseudo-code,  the  iMRI  slice  image  is  the  reference  slice,  the  slice  image  extracted  from  the  high-resolution  MRI 
volume  is  the  reformatted  slice,  and  the  final  reformatted  slice  is  the  registered  slice .  We  used  a  multi-resolution  approach 
and  performed  registration  from  low  to  high  resolution.  We  used  MI  at  the  highest  resolution  because  it  gave  a  more  robust 
solution,  and  we  used  CC  at  the  lower  resolutions  because  it  gives  few  local  optima  and  because  it  calculates  faster  than  MI. 
We  created  a  method  to  avoid  local  minima  by  restarting  the  registration  with  randomly  perturbed  parameters  obtained  from 
a  uniform  distribution  about  the  very  first  initial  guess.  The  algorithm  restarts  until  the  absolute  CC  between  the  reference 
and  registered  images  is  above  a  threshold  or  the  maximum  number  of  restarts  is  reached.  Absolute  CC  is  used  for  the  restart 
test  rather  than  MI  because  it  has  a  well-defined  range  between  0  and  1  and  because  it  provides  an  independent  check  of  the 
MI  result. 

We  used  rigid  body  transformation  (three  translations  and  three  rotations)  and  trilinear  interpolation  as  described 
previously.1  For  optimization,  we  used  the  downhill  simplex  method  of  Nelder  and  Mead  22  and  the  Powell  method,23  but  we 
prefer  the  former  method  as  described  later.  Optimization  of  similarity  ends  either  when  the  maximum  number  of 
calculations  is  reached  (typically  500)  or  the  fractional  change  in  similarity  function  is  smaller  than  a  tolerance  (typically 
0.001).  Our  very  first  initial  guess  at  the  lowest  resolution  is  all  zeros  for  the  3  displacements  and  3  angles.  Based  on  our 
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Set  an  initial  reformatted_slice  in  high-resolution  MR  volume  and  CC_thresholds 
DO  registration  FROM  lowest  resolution  TO  highest  resolution  BEGIN 
Resample  volume  and  slice  to  %,  Yi  or  full  number  of  voxels  along  linear  dimension 
Initialize  registration_results  and  number_restarts  to  zero  for  restarting  registrations 

REPEAT 

Optimize  similarity  (CC*  or  MI**)  between  the  reference  slice  and  reformatted_slice 

1.  Transform  the  initial  reformatted_slice  and  interpolate  to  get  a  new  reformatted_slice 

2.  Calculate  similarity  between  the  reference  slice  and  the  new  reformatted_slice 

3.  Repeat  1  and  2  until  meeting  function  tolerance  or  maximum  iteration  number 
Calculate  CC  between  the  reference  slice  and  the  optimal  reformatted_slice 
Record  CC,  MI**  values  and  transformation  parameters  to  registration_j*esults 
Perturb  the  initial  transformation  parameters: 

initiaLtransformation_parameters  -  initial_transformation_parameters  +  random  •  factor 
Trace  the  number  of  restarts:  number_restarts  =  number_restarts  +  1 
UNTIL  (CC  >  CC_threshold)  OR  (number_restarts  >  maximum_re starts) 

Select  the  final  transformation_parameters  based  on  CC*  or  MI**  among  registrationjresults 
Scale  the  parameters  and  assign  to  initial_transformation_parameters  of  next  higher  resolution 
END _ _ _ _ _ 

Figure  1.  Registration  algorithm.  Capital  bold  words  are  computer  language.  The  outer  loop  from  DO  to  END 
gives  the  multi-resolution  approach.  The  inter  loop  from  REPEAT  to  UNTIL  is  for  restarting  registration. 
Registrationjresults  and  number_restarts  are  used  to  store  temporary  values  in  the  program.  See  text  for  details. 

CC  is  used  at  lower  resolutions,  lA  or  Yi  number  of  voxels. 

**  MI  is  used  only  at  high  resolution,  full  number  of  voxels. 

experience,  we  set  the  CC  thresholds  at  0.60, 0.65  and  0.70,  and  the  maximum  numbers  of  restarts  at  25,  15,  and  5,  from  low 
to  high  resolution,  respectively. 

In  Figure  1,  there  are  several  differences  between  low  and  high  resolutions.  At  lower  resolutions,  we  resample  the 
volume  and  slice  to  1/4  or  1/2  size  along  the  linear  dimension.  We  optimize  the  CC  between  reference  and  reformatted  slices 
to  obtain  optimal  transformation  parameters.  We  record  important  results  such  as  all  CC  optimized  values,  the  number  of 
restarts,  and  the  transformation  parameters  following  an  optimization.  At  the  end  of  either  of  these  resolutions,  we  select  the 
transformation  parameters  that  have  the  maximum  CC  value.  We  then  scale  the  parameters  and  assign  them  to  be  initial 
values  at  the  next  higher  resolution.  At  the  highest  resolution,  MI  instead  of  CC  is  chosen  to  be  the  similarity  measurement 
method,  and  we  select  the  final  transformation  parameters  with  maximum  MI  instead  of  CC. 

There  are  several  preprocessing  details.  The  input  MRI  volumes  are  256  x  256  x  128  voxels  that  have  an  almost  isotropic 
size  over  a  field  of  view  covering  the  whole  pelvis.  Isotropic  voxels  are  created  using  3D  linear  interpolation  or  higher  order 
interpolation  methods.1 


3.  EXPERIMENTAL  METHODS 


3.1.  Data  Acquisition 

All  MRI  volumes  were  acquired  using  a  1.5  T  Siemens  MRI  system  (Magnetom  Symphony,  Siemens  Medical  Systems 
in  Erlangen,  Germany).  An  8-element  phased  array  body  coil  was  used  to  ensure  coverage  of  the  prostate  with  a  uniform 
sensitivity.  Typically  two  anterior  and  two  posterior  elements  were  enabled  for  signal  acquisition.  We  used  a  3D  PSIF 
sequence  with  9.4/5.0/60  (TR/TE/flip)  yields  160  x  256  x  128  voxels  over  a  219  x  350  x  192-mm  rectangular  FOV  and 
1.37  x  1.37  x  1.5-mm  voxels  oriented  to  give  the  highest  resolution  for  transverse  slices.  There  is  over-sampling  at  31%  in 
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the  slice  direction  to  reduce  aliasing  artifacts.  This  sequence  gave  excellent  image  contrast  for  the  prostate  and  its 
surroundings. 

3.2.  Creation  of  Images  for  Testing 

To  test  the  ability  of  the  slice-to-volume  (SV)  registration,  we  obtained  high-resolution  MRI  volumes  and  simulated 
iMRI  images  by  adding  noise  and  receive  coil  inhomogeneity.  We  compared  SV  registration  to  volume-to-volume  (W) 
registration  results. 

We  simulated  iMRI  slice  images.  We  averaged  3  slices  together  to  simulate  an  iMRI  4-mm  thick  slice.  We  used  a 
homogenous  phantom  to  measure  the  SNR  of  iMRI  images  on  our  0.2  T  open  magnet  system  (Magnetom  Open,  Siemens 
Medical  Systems  in  Erlangen,  Germany).  We  used  simulated  iMRI  images  to  test  the  dependence  of  registration  on  noise 
levels.  Figure  2  shows  the  simulated  iMRI  slice  images. 

3.3.  Registration  Experiments 

We  acquired  high-resolution  MR  volumes  from  three  volunteers.  Each  has  two  pairs  of  volumes  for  registration.  We 
extracted  three  slices  from  one  volume  and  used  these  slices  to  simulate  an  iMRI  image.  We  tested  multiple  ways  to  acquire 
slices  for  SV  registration.  First,  we  used  transverse,  sagittal  and  coronal  slices  for  registration,  respectively.  The  objective 
was  to  optimize  the  slice  orientation  for  image  guidance.  Second,  we  extracted  slices  from  different  positions  such  as 
centered  at  the  prostate,  and  above  and  below  the  prostate  by  35  mm.  This  experiment  was  performed  to  investigate  the  range 
for  reliable  slice  registration.  Third,  we  tested  the  dependence  of  the  registration  on  noise  levels. 


Figure  2.  Simulated  iMRI  images.  Images  on  the  left,  (a),  (c)  and  (e),  are  the  high-resolution  MR  images  in  transverse, 
coronal  and  sagittal  planes,  respectively.  Images  on  the  right  are  corresponding  simulated  iMRI  images  with  SNR  =  8. 
The  images  are  of  volunteer  V2. 
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Figure  3.  SV  registration  using  slices  at  different  orientations.  The  error  metric  is  voxel  displacement  between  the  SV  and 
W  results.  Plotted  are  mean  errors  as  well  as  maximums  and  minimums  that  show  the  spread  of  the  data.  SI,  S2,  and  S3 
refer  to  volunteers.  Each  volunteer  has  two  volume  pairs.  For  each  pair,  five  transverse  slices  centered  at  the  prostate  were 
extracted  from  one  volume  and  registered  to  the  other.  The  same  procedures  were  applied  to  coronal  and  sagittal  slices. 


3.4.  Evaluation  Methods 

To  test  the  program,  we  transformed  a  reference  volume  using  known  parameters  to  obtain  a  digital  phantom.  We 
extracted  slices  from  the  digital  phantom  and  registered  them  to  the  reference  volume.  Because  the  transformation  was 
known,  we  could  validate  the  performance  of  the  software. 

Our  standard  evaluation  method  was  to  compare  SV  and  W  registration.  The  W  registration  accuracy  was  previously 
evaluated  to  be  on  the  order  of  one  voxel  (1.37  mm)  14.  We  defined  a  volume  of  interest  (VOI)  just  covering  the  prostate  and 
applied  the  transformations  obtained  by  W  and  SV  registrations  to  voxels  within  the  VOI.  We  calculated  the  3D 
displacement  on  a  voxel-by-voxel  basis  over  the  two  transformed  VOIs.  The  mean  voxel  displacement  was  used  as  our 
metric  of  SV  registration  error. 


4.  RESULTS  AND  DISCUSSIONS 

More  than  300  SV  registration  experiments  were  performed  under  a  variety  of  conditions  expected  for  applications  in 
iMRI-guided  treatment  of  the  prostate.  First,  we  report  the  effect  of  slice  orientation  and  position  on  registration.  Second,  we 
describe  the  results  associated  with  image  noise.  At  the  end,  we  describe  some  details  on  the  algorithm  implementation. 
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Figure  4.  SV  registration  using  transverse  slices  at  different  positions.  Relative  to  the  prostate,  five  iMRI  slices  each  were 
extracted  near  its  center,  «  35  mm  above  its  center,  and  ~  35  mm  below  its  center.  Other  details  are  given  in  Figure  3, 
Slices  at  the  prostate  center  work  best. 


4.1.  Slice  Orientation  and  Position 

Figure  3  shows  registration  results  for  single  slices  oriented  in  different  directions.  All  slices  intersect  the  prostate. 
Transverse  slices  give  the  lowest  average  error  across  the  three  volunteers,  0.39  ±0.19  mm.  The  reason  for  this  result  is  that  a 
transverse  slice  contains  more  anatomical  structures  than  do  slices  in  other  orientations.  A  transverse  slice  also  excludes  the 
bladder  and  portions  of  the  rectum  that  can  deform  and  create  inconsistent  matches  for  registration.  The  following  analyses 
are  all  based  on  transverse  slices. 

Figure  4  shows  registration  results  for  transverse  slices  at  different  heights  relative  to  the  prostate.  Slices  centered  on  the 
prostate  produced  the  best  results  with  the  displacement  error  always  being  less  than  1.0  mm.  On  the  one  hand,  slices 
centered  at  the  prostate  are  good  because  they  include  an  abundance  of  bony  structures  that  provide  adequate  information  for 
registration.  Slices  above  the  prostate  include  the  bladder,  which  can  deform  and  stress  the  registration  algorithm.  Slices 
below  the  prostate  mainly  contain  fatty  regions  from  the  hips  that  can  also  deform  and  have  less  information  for  registration. 

4.2.  Noise  Level  and  Robustness 

We  performed  preliminary  experiments  using  different  noise  levels  in  the  simulated  iMRI  images.  When  SNR  values 
were  30,  15  and  5,  registration  errors  were  always  less  than  0.9  mm  for  three  volunteers  in  all  conditions.  For  more  than  200 
registration  experiments,  the  algorithm  was  very  robust  and  never  failed  even  with  these  large  noise  values.  We  believe  the 
algorithm  to  be  insensitive  to  noise. 

4.3.  Program  Implementation 

We  implemented  the  registration  program  using  IDL  (Interactive  Data  Language,  Research  System  Inc.,  Boulder,  CO.). 
The  time  for  an  SV  registration,  typically  40  to  90  sec  on  a  Pentium  III,  800  MHz  CPU,  with  512  Mbytes  of  memory,  could 
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be  improved  using  optimized  C  code  rather  than  IDL.  The  simplex  optimization  method  worked  faster  than  the  Powell 
method  in  our  implementation.  This  result  is  contrary  to  our  previous  experience.1  We  think  that  this  phenomenon  occurred 
because  the  Powell  method  depends  on  the  order  that  parameters  are  optimized.  Because  we  do  not  require  preprocessing  to 
determine  initial  values,  there  are  no  clear  choices  for  which  parameters  to  optimize  first.  Each  call  to  the  Simplex 
optimization  for  restarts  or  optimization  at  a  finer  resolution  resulted  in  50  to  105  similarity  evaluations  before  the  tolerance 
value  (0.001)  was  reached. 

The  restarting  and  multi-resolution  features  are  important.  Using  normal  clinical  transverse  images  covering  the  prostate, 
the  algorithm  always  gave  very  nearly  the  same  transformation  parameters  (less  than  0.01  voxels  and  0.01  degrees)  using  a 
wide  variety  of  initial  guesses.  The  multi-resolution  approach  enabled  the  program  to  get  close  to  the  final  value  quickly 
because  of  the  reduced  number  of  calculations.  Althought  serious  motion  artifacts  may  stress  the  registration,  our  experience 
is  that  these  can  be  controlled  with  appropriate  subject  compliance  and  with  the  proper  acquisition  technique.  Organ  motion 
and  deformation  from  one  acquisition  to  the  next  are  other  factors  that  can  affect  the  performance  of  the  algorithm,  especially 
when  subject  is  in  a  different  position  or  there  is  rectal  and  bladder  filling.  14  Again,  our  experience  is  that  these  errors  can 
be  controlled.  14 


5.  CONCLUSION 

The  automatic  slice-to-volume  mutual  information  registration  algorithm  was  quite  robust  for  transverse  slice  images 
covering  the  prostate.  There  were  no  registration  failures  in  over  200  experiments  on  MRI  images  without  obvious  artifacts. 
The  registration  error  of  <  1 .0  mm  should  be  sufficiently  accurate  to  aid  in  the  application  of  iMRI-guided  minimal  invasive 
thermal  ablation  of  prostate  cancer.  We  are  beginning  to  explore  these  applications  in  clinical  procedures  and  animal 
experiments. 
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iMRI-Guided  Thermal  Ablation 


•  Open  magnet 
allows  access 
to  patient 

•  RF  probe 
inserted  under 
iMRI  guidance 

•  High  frequency 
electric  current 
heats  and 
destroys  tumor 


Goal 


Include  information  from  various 
modalities  such  as  SPECT  and  MRI 
in  iMRI-guided  treatment  paradigm 


Image  of  the  year  2000,  Society  of  Nuclear  Medicine,  Created  by  Dr  Z.  Lee,  CWRU 
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Image  Registration 


Slice  to  Volume  Registration 

•  Register  iMRI  thick-slice  image 
to  high  resolution  MRI  volume 

•  Rigid  body  transformation 

-  3  translations  and  3  rotations 

•  Optimization  method 

-  Downhill  simplex 

•  Algorithm  features 

-  Mutual  information 

-  Multi-resolution  approach 

-  Restarting  algorithm 


MRI 

Volume 
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Mutual  Information  5'  Mutual  Information 
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Prostate  MR  Images 


High  resolution  Simulated  iMRI  Actual  iMRI 
MR  images  images  images 


Volume  Registration  Standard 
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Distance  between  Bony  Landmarks 
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Evaluation:  Voxel  Displacement 

3D  voxel  displacement  over  volume  of  interest 
(VOI)  between  S-V  and  V-V  registration 


Volume-volume  registration 


Experiments _ 

•  800  slice-to-volume  registration 
using  simulated  iMRI  images 

•  450  slice-to-volume  registration 
using  actual  iMRI  images 

•  Registration  evaluation  using  V-V 
results  and  visual  inspections 

•  Image  fusion  and  visualization 
software  for  iMRI  guidance 
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Slice  Orientation 


Transverse  Coronal  Sagittal 

slices  slices  slices 


Transverse  is  best 
0.4  ±  0.2  mm 


Transverse 


Coronal 


Sagittal 
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Effect  of  Inhomogeneity 


Registration  is  not  sensitive  to  coil  position 
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Results  of  Actual  iMRI 
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Registration  Error  (mm) 


Effect  of  Slice  Thickness 
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iMRI  slice  with  a  thickness  of  5  mm  is  best. 


Phantom  Experiments 
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Image  Guidance 
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Discussion 


•  Slice  to  volume  registration  is  very  robust 
and  accurate  ( « 1  mm)  for  transverse 
slices  covering  the  prostate. 

•  SPECT  and  high  resolution  MRI  image  can 
be  included  into  interventional  MRI  guided 
treatment  with  improved  tumor  targeting. 

•  Dog  experiments  are  being  developed  to 
verify  the  iMRI-guided  radiofrequency 
thermal  ablation  of  the  prostate  cancer. 
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Image  Modalities 


Before 

Treatment 


During 

Treatment 


Image  Registration 


High  Resolution  MR  Volume 


SPECT  Image  Volume 
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Registration  Algorithm 

•  Similarity  Measures 
-Normalized  mutual  Information  (NMI) 
-Mutual  information  (Ml) 

-Correlation  coefficient  (CC) 


Registration  Algorithm 

•  Similarity  Measures 

•  Interpolation  methods 
-Normalized  sine  interpolation 
-Tri-linear  interpolation 
-Nearest  neighbor  interpolation 
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Registration  Algorithm 

•  Similarity  Measures 

•  Interpolation  methods 

•  Optimization  methods 
-Downhill  simplex 
-Power  method 


Registration  Algorithm 

•  Similarity  Measures 

•  Interpolation  method 

•  Optimization  method 

•  Other  features 
-Restarting  algorithm 
-Multi-resolution  approach 
-Rigid  body  transformation 
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Similarity  Surface 


Similarity  measure  as  a  function  of 
translations  between  two  images. 


Similarity  Surfaces  at  Low  Resolution 
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Similarity  Surfaces  at  Low  Resolution 
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17 

Multi-resolution  and  Multi-similarity 
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Image  Acquisitions 

•  6  volunteers  and 

3  patients 

•  22  MR  volumes 

from  1.5  T  MR 

•  12  iMRI  volumes 

from  0.2  T  MR 

•  150  real  iMRI 

slices  from  0.2  T 
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Prostate  MR  Images 


High  resolution  Simulated  iMRI  Real  iMRI 
MR  images  images  images 
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Simulation  of  RF  Needle 


Registration 

Evaluation 
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Volume  Registration  Standard 
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Distance  between  Bony  Landmarks 
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Volume-to-Volume  Registration 

•  Distance  between  bony 
landmarks  <1.6  mm 

•  Prostate  3D  centroids  <1.2  mm 

•  Prostate  overlays  very  closely 

Volume-to-volume  registration  is 
accurate  and  provides  a  standard 
for  slice-to-volume  registration 
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Evaluation:  Voxel  Displacement 

3D  voxel  displacement  over  volume  of  interest 
(VOI)  between  S-V  and  V-V  registration 


Slice-volume  registration 


Volume-volume  registration 
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Registration 

Results 
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Slice  Orientation 


Transverse  is  best 
0.4  ±  0.2  mm 


Transverse 


Coronal 
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Effect  of  Noise  Level 
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360  slice-to-volume  registration 
using  simulated  iMRI  images 
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SNR=20 


SNR=10 


SNR=5 


SNR=3 


Registration  is  robust  for  noisy  images 


30 


Results  of  Real  iMRI 
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Image  Guidance 
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Discussion 


•  Slice  to  volume  registration  is  very  robust 
and  accurate  ( «  1  mm)  for  transverse 
slices  covering  the  prostate. 

•  SPECT  and  high  resolution  MRI  image  can 
be  included  into  interventional  MRI  guided 
treatment  with  improved  tumor  targeting. 

•  Dog  experiments  are  being  developed  to 
verify  the  iMRI-guided  radiofrequency 
thermal  ablation  of  the  prostate  cancer. 


213 


Acknowledgements 

DOD  Grant : 

•  DAMD  17-02-1-0230  to  Baowei  Fei 
NIH  Grants  : 

•  R01  CA84433-01  to  David  L.  Wilson 

•  R33CA81 44-01  to  Jeffery  L.  Duerk 


214 


